SSIS Dynamic Connections Part 2

SSIS Dynamic Connections Part 1

If you didn’t see part one I suggest you click the link above as I will be continuing from there.

For Part 2 we are going to show how to create a connection and then change the properties of the connection to process multiple files. So to set this up I created a Table called FilesToProcess in a database called ConnDemo.

Create
Table FilesToProcess

(

FileName
nvarchar(50),

FileLocation nvarchar(150)

)

With the following values.

I also created a table called StgImportFiles to import the files into.

CREATE
TABLE StgImportFiles

(

Col1 NVARCHAR(50),

Col2 NVARCHAR(50),

Col3 NVARCHAR(50)

)

What we are going to do is query this table to get all the files to process and then process each File by inserting them into the same table. Now remember to do this all the files have to have the same format.

I am going to expand on the Package from Part1 and just change the database name in the configuration file to be ConnDemo and then I will doing the following tasks.

1. Add variables to store the file location and name as well as the return values of the query.
2. Add Execute SQL Task to get the values from the FilesToProcess Table
3. Add Foreach Loop Container to process the files
4. Add DataFlow task to import the data.

The variables I am going to add are

1. FileLocation – Store the Location of the File
2. FileName – Store the Name of the File
3. FileListResults – note that this is a Data Type of Object as this is required

Now I am adding an Execute SQL Task and Naming it “Get Files To Process”, setting the connection to be the DemoSQLConnection, setting the ResultSet to Full result set, then adding the following sql statement.

select FileLocation,FileName

from FilesToProcess

Then add a Result set and map it to the FileListResults variable. Your Execute SQL task should look like below and You must set the Result Name to 0 as that is the position of the result set.

Now we add the Foreach Loop Container and add the collection with enumerator “Foreach ADO Enumerator” and the map to the FileListResults variable. Then add the variable Mappings of FileLocation and FileName and should look like the screen shots below.

Next we add a Data Flow inside of the Foreach Loop and Create a File Connection Manager called Importfiles we will point this initially to one of the files we want to import so that we can properly set the format of the files.

Now that we have the Source File we can modify the connection string expression of the File to be @[User::FileLocation] + @[User::FileName]. This is what will now change the path of the file as each row of the returning query goes through the Foreach Loop.

The last step is to add to the DataFlow a Source pointing the File Connection Manger Imporfile and a destination pointing to the DemoSQLConnection and selecting the StgImportFiles table. Your package should now look like the below.

If you run the package the values in the StgImportFiles table should be there.

This process can be very useful if you receive multiple files in the same format that need to be imported and don’t want to write separate packages for each file. I have used this approach where we needed to import files for over 200 customers and the layout was a standard one that we had created and I only created one package to import all 200 files.