Use Powershell for Incremental Load into different tables

This topic contains 1 reply, has 2 voices, and was last updated by  Rob Simmers 3 weeks, 1 day ago.

  • Author
    Posts
  • #100138

    Possibility1234
    Participant

    My question and requirement are listed below:
    1. I have many files in txt and csv formats(files A, B, C, D)
    2. These files will load SQL tables (A, B, C, D)
    3. These files have different metadata and unique keys
    4. These files will be loaded to target tables as full load the first time
    –Subsequent files(A,B,C,D) will be loaded on incremental basis:
    —-If the primary key column already exist, update non-key columns
    —-if the primary key column does not exist, insert as new rows

    What I read on this link seems to touch on it barely:

    What I have done in the past(I removed the code so it does not cause confusion–Where I passed all my entries as parameters)

    Create an array list from a csv file which has the entries for the files name(A, B,C,D)
    Loop for each file name and process:
    My challenge is, assuming the first load is a full load, subsequent load will either update or insert.
    1. How would I get all the columns as parameters
    2. Pick the the key column(s)
    3. Compare file entries to existing equivalent table
    4. If primary key exist, update none-key column but if primary key does not exist, insert into target table as new rows

    Thank you

  • #100156

    Rob Simmers
    Participant

    Can you be clear on what you need assistance with? Your post is a statement, not a question.

You must be logged in to reply to this topic.