Convert LOG file to CSV

Tagged: 

This topic contains 8 replies, has 3 voices, and was last updated by Profile photo of Venkata Kalyan Venkata Kalyan 5 months, 2 weeks ago.

  • Author
    Posts
  • #42839
    Profile photo of Venkata Kalyan
    Venkata Kalyan
    Participant

    Hi,
    I have a logfile in a folder. I am searching a word (externalsystemorderservice) in that logfile and converting those lines to CSV.
    Below is the script:

    $log = Get-ChildItem -Path C:\temp\*.log
    $col = {@()}.invoke()
    switch -Regex -File $log
    {
    '#Fields.*' {$fields = $_}

    '\d+.*externalsystemorderservice.*' {$col.add($Matches[0])}
    }

    $fields = $fields -replace '#Fields:'
    $fields = $fields.trim() -split ' '

    $col -replace ' ',',' | convertfrom-csv -Header $fields |
    Export-Csv -Path C:\temp\final.csv -NoTypeInformation

    I am able to convert for one log, but i have 7 logfiles in a folder. I am not able to perform this operation.
    Can some please help me on this.

    NOTE: logfiles names will be different, but it has extension.log

    -Kalyan

  • #42882
    Profile photo of Richard Diphoorn
    Richard Diphoorn
    Participant

    You could simply use a foreach loop to perform your action on every logfile:

    $logFiles = Get-ChildItem -Path 'C:\temp' -Filter *.log
    
    foreach ( $logFile in $logFiles ) {
    
    your code goes here
    
    }
    
  • #42908
    Profile photo of Venkata Kalyan
    Venkata Kalyan
    Participant

    Hi,
    Thanks, I tried the script. It is doing some thing, but the exact result is not coming.
    Ex: I have five IIS logs files in a folder & running this script as below:

    NOTE: Logfile names: u_ex160609.log, u_ex160610.log, u_ex160611.log, u_ex160612.log, u_ex160613.log
    These keep on changing as per date.

    $logFiles = Get-ChildItem -Path 'C:\temp\Test' -Filter *.log
    foreach ( $logFile in $logFiles ) {

    $col = {@()}.invoke()
    switch -Regex -File $logFile
    {
    '#Fields.*' {$fields = $_}
    '\d+.*externalsystemorderservice.*' {$col.add($Matches[0])}
    }

    $fields = $fields -replace '#Fields:'
    $fields = $fields.trim() -split ' '
    $col -replace ' ',',' | convertfrom-csv -Header $fields |
    Export-Csv -Path C:\temp\Test\final_dsfprdapp711.csv -NoTypeInformation
    }

    But, the date column is showing with latest log file date.

    Any suggestions?

    -Kalyan

  • #42910
    Profile photo of random commandline
    random commandline
    Participant

    Make this change. A csv is created for each log file.

    Export-Csv -Path C:\temp\Test\final_$($logFile.basename).csv -NoTypeInformation
    
  • #42953
    Profile photo of Richard Diphoorn
    Richard Diphoorn
    Participant

    Do you want to have every single line in a separate csv file?

  • #42960
    Profile photo of Venkata Kalyan
    Venkata Kalyan
    Participant

    Hi,
    I am planning to have all log files in one CSV file.
    But the headers column should be only one.

    -Kalyan

  • #43022
    Profile photo of Venkata Kalyan
    Venkata Kalyan
    Participant

    Hi,
    To be more precise, requirement goes like the below:
    I have seven logfiles.
    > I need to go through each logfile and find a string (externalsystemorderservice) and get that line with limited headers (date,time,s-computername,s-ip,time-taken) to csv file.

    All logfiles data to a single CSV file.

    I am able to do for one logfile. But it comes to seven logfiles, I am not able to do.

    -Kalyan

  • #43059
    Profile photo of random commandline
    random commandline
    Participant

    What does your first matched line and fields look like? You will need to specify headers for the lines in your collection before exporting.
    This is an example.

    $col | ConvertFrom-Csv -Header 'cs','cs-host','sc-status','sc-bytes' -Delimiter ' ' |
    Export-Csv -Path C:\temp\Test\final_dsfprdapp711.csv -NoTypeInformation -Append
    
    # Results
    # cs                              cs-host                sc-status             sc-bytes             
    # --                                -------                ---------             --------             
    # https://domain.foo...  domain.foo           301                   581  
    
  • #43967
    Profile photo of Venkata Kalyan
    Venkata Kalyan
    Participant

    Thanks. Let me try.

    -Kalyan

You must be logged in to reply to this topic.