Use Powershell for Incremental Load into different tables

Welcome Forums General PowerShell Q&A Use Powershell for Incremental Load into different tables

This topic contains 1 reply, has 2 voices, and was last updated by

 
Participant
6 months, 2 weeks ago.

  • Author
    Posts
  • #100138

    Participant
    Points: 0
    Rank: Member

    My question and requirement are listed below:
    1. I have many files in txt and csv formats(files A, B, C, D)
    2. These files will load SQL tables (A, B, C, D)
    3. These files have different metadata and unique keys
    4. These files will be loaded to target tables as full load the first time
    –Subsequent files(A,B,C,D) will be loaded on incremental basis:
    —-If the primary key column already exist, update non-key columns
    —-if the primary key column does not exist, insert as new rows

    What I read on this link seems to touch on it barely:
    https://powershell.org/forums/topic/csv-to-sql-database-insertupdate/

    What I have done in the past(I removed the code so it does not cause confusion–Where I passed all my entries as parameters)

    Create an array list from a csv file which has the entries for the files name(A, B,C,D)
    Loop for each file name and process:
    My challenge is, assuming the first load is a full load, subsequent load will either update or insert.
    1. How would I get all the columns as parameters
    2. Pick the the key column(s)
    3. Compare file entries to existing equivalent table
    4. If primary key exist, update none-key column but if primary key does not exist, insert into target table as new rows

    Thank you

  • #100156

    Participant
    Points: 158
    Helping Hand
    Rank: Participant

    Can you be clear on what you need assistance with? Your post is a statement, not a question.

The topic ‘Use Powershell for Incremental Load into different tables’ is closed to new replies.