Comparing 2 CSV files

Welcome Forums General PowerShell Q&A Comparing 2 CSV files

Viewing 5 reply threads
  • Author
    Posts
    • #226686
      Participant
      Topics: 2
      Replies: 2
      Points: 50
      Rank: Member

      Hello,

      I have 2 CSV files, let’s call them old.csv and new.csv – it’s an automated file spat out by our HRIS every hour or so and has about 1000 rows and about 20 columns, and I’m comparing the new one with the one that’s an hour old.

      I need to create a third file called changes.csv which lists the entire row for any row where any of the fields in that row has a different value in the new.csv than in the old.csv, and also any new.csv row that isn’t in the old.csv

      Both old.csv and new.csv have a header row, and there’s a unique field in the EmployeeID column.

      What I’ve written so far doesn’t really bear posting here, as it’s tortuous and clunky, and I haven’t tried running it yet.

      As with many scriplets I’m sure someone somewhere has had the same need I do and thrown something together that works and then gets streamlined and then gets simplified until it’s a work of art, haha. So I was wondering if anyone here has something approaching a one-liner I can incorporate into our custom reporting scripts that creates the changes.csv I’m looking for.

      If not, any pointers/suggestions/gotchas would be most welcome.

      Thanks!

    • #226692
      Participant
      Topics: 7
      Replies: 2458
      Points: 6,439
      Helping Hand
      Rank: Community MVP

      What I’ve written so far doesn’t really bear posting here, as it’s tortuous and clunky, and I haven’t tried running it yet.

      So we have to start from scratch to create something for your, right? You could at least post a few sanitized but still representative rows of the source data you’re dealing with.

      As with many scriplets I’m sure someone somewhere has had the same need I do and thrown something together that works ….

      Did you try to search for it? I’m pretty sure you could have find something adaptable in StackOverflow or even here in this forum or in the PowershellGallery.

      Have you tried the cmdlet Compare-Object?

    • #226695
      Participant
      Topics: 2
      Replies: 2
      Points: 50
      Rank: Member

      Compare-Object was the way to go, thanks. I was comparing each entry in the hashtables in two foreach loops. Longwinded and tortuous, as I said.

    • #226713
      Participant
      Topics: 0
      Replies: 1
      Points: 20
      Rank: Member
    • #226725
      Participant
      Topics: 0
      Replies: 1
      Points: 5
      Rank: Member

      AMGPoker adalah situs Judi Online terpercaya se-Asia
      Kami juga menyediakan bonus untuk anda seperti :
      * Bonus new member
      * Bonus rollingan
      * Bonus referral
      WA : +855 1070 2288
      http://ref.mov168.xyz/?slug=“jty6r”&

    • #226761
      Participant
      Topics: 17
      Replies: 1872
      Points: 3,640
      Helping Hand
      Rank: Community Hero

      I’ve actually had similar requirements in the past trying to create reports that showed changes (e.g. deltas) from one HR sync to another. The goal was to get who and what the current and old value was. Compare-Object tells us something is different, but not exactly what changed, only the new value. Be curious if any other experts know a better way to get this information:

      Output:

      I would create PowerBi reports from this data, which would also include a Date to see what happened during each sync. The function could be made more efficient by possibly running a Compare-Object to get only changed rows and then process the properties, but curious how long this would take against 1000 rows * 20 properties. Another option is to use SQL as well:

      Different Ways to Compare SQL Server Tables Schema and Data

Viewing 5 reply threads
  • The topic ‘Comparing 2 CSV files’ is closed to new replies.