I have a very large text file with well over 1 million lines in it that I need to do some data analyst on.
What I need to accomplish is take this data file and get the data from the lines where the text in column 2-3 contains "NN" or"nn"
Then from that sub set I need to get the values from columns 38-56 and 167-185.
I then need to remove any duplicated that are found in the 167-185 columns.
I then need to compare what remains from the 167-185 columns to the data from the 38-56 columns and return the values from that data set where there are no matches or null data.
I have been poking around the net for a place to start but so far I am coming up short and I have very little time to get this done so I am hoping the the PS community can help me out.