batch rename csv headers

Tagged: 

This topic contains 3 replies, has 2 voices, and was last updated by Profile photo of Tim Pringle Tim Pringle 11 months ago.

  • Author
    Posts
  • #33758
    Profile photo of liri321
    liri321
    Participant

    I'm trying to change headers of a very large csv file .

    All headers have 'some_string' in each header name , like "some_string.header_name"

    My csv file is big , more than 200G and has more than 200 columns

    I'll very appricciate for any idea how to change headers fast and effectively !

  • #33759
    Profile photo of Tim Pringle
    Tim Pringle
    Participant

    Hi liri321

    Leaving aside why you're using a CSV file for a 200GB file for the moment, the use of StreamReader and StreamWriter is probably going to be your best bet.

    Question I have though is do you have a fixed first line that you want to put in the CSV file, overwriting the first line, or do you actually want to do a search and replace of items in the first line?

    If the latter, can you give more details?

  • #33765
    Profile photo of liri321
    liri321
    Participant

    Hi Tim

    I'd like to overwrite the csv first line . I thought about serach and replace strings in header . What solution is faster and effectively ? Overwriting or serach/replace of "somestring" ?

    Thanks !

  • #33777
    Profile photo of Tim Pringle
    Tim Pringle
    Participant

    If you already know the text that is to go in the first line, i think it's a better method just to put it as the first line.

    I don't have code to hand, but take a look at the StreamReader and StreamWriter. The method i'd imagine for use would be :

    – Open output file with StreamWriter
    – Write what the header is supposed to be
    – Open input file
    – Read the first line
    – Loop through the rest of the file, performing a read operation of the input file followed by a write operation to the output file until you get to the end of the file.
    -Close the streams

    Because you read the first line of the input file, but do nothing, that means it will be ignored when you are creating the new output file.

You must be logged in to reply to this topic.