- January 12, 2016 at 2:43 am #33758
I'm trying to change headers of a very large csv file .
All headers have 'some_string' in each header name , like "some_string.header_name"
My csv file is big , more than 200G and has more than 200 columns
I'll very appricciate for any idea how to change headers fast and effectively !January 12, 2016 at 2:51 am #33759
Leaving aside why you're using a CSV file for a 200GB file for the moment, the use of StreamReader and StreamWriter is probably going to be your best bet.
Question I have though is do you have a fixed first line that you want to put in the CSV file, overwriting the first line, or do you actually want to do a search and replace of items in the first line?
If the latter, can you give more details?January 12, 2016 at 4:38 am #33765
I'd like to overwrite the csv first line . I thought about serach and replace strings in header . What solution is faster and effectively ? Overwriting or serach/replace of "somestring" ?
Thanks !January 12, 2016 at 6:26 am #33777
If you already know the text that is to go in the first line, i think it's a better method just to put it as the first line.
I don't have code to hand, but take a look at the StreamReader and StreamWriter. The method i'd imagine for use would be :
– Open output file with StreamWriter
– Write what the header is supposed to be
– Open input file
– Read the first line
– Loop through the rest of the file, performing a read operation of the input file followed by a write operation to the output file until you get to the end of the file.
-Close the streams
Because you read the first line of the input file, but do nothing, that means it will be ignored when you are creating the new output file.
You must be logged in to reply to this topic.