Convert LOG file to CSV

This topic contains 8 replies, has 3 voices, and was last updated by  Venkata Kalyan 1 year, 4 months ago.

  • Author
    Posts
  • #42839

    Venkata Kalyan
    Participant

    Hi,
    I have a logfile in a folder. I am searching a word (externalsystemorderservice) in that logfile and converting those lines to CSV.
    Below is the script:

    $log = Get-ChildItem -Path C:\temp\*.log
    $col = {@()}.invoke()
    switch -Regex -File $log
    {
    '#Fields.*' {$fields = $_}

    '\d+.*externalsystemorderservice.*' {$col.add($Matches[0])}
    }

    $fields = $fields -replace '#Fields:'
    $fields = $fields.trim() -split ' '

    $col -replace ' ',',' | convertfrom-csv -Header $fields |
    Export-Csv -Path C:\temp\final.csv -NoTypeInformation

    I am able to convert for one log, but i have 7 logfiles in a folder. I am not able to perform this operation.
    Can some please help me on this.

    NOTE: logfiles names will be different, but it has extension.log

    -Kalyan

  • #42882

    Richard Diphoorn
    Participant

    You could simply use a foreach loop to perform your action on every logfile:

    $logFiles = Get-ChildItem -Path 'C:\temp' -Filter *.log
    
    foreach ( $logFile in $logFiles ) {
    
    your code goes here
    
    }
    
  • #42908

    Venkata Kalyan
    Participant

    Hi,
    Thanks, I tried the script. It is doing some thing, but the exact result is not coming.
    Ex: I have five IIS logs files in a folder & running this script as below:

    NOTE: Logfile names: u_ex160609.log, u_ex160610.log, u_ex160611.log, u_ex160612.log, u_ex160613.log
    These keep on changing as per date.

    $logFiles = Get-ChildItem -Path 'C:\temp\Test' -Filter *.log
    foreach ( $logFile in $logFiles ) {

    $col = {@()}.invoke()
    switch -Regex -File $logFile
    {
    '#Fields.*' {$fields = $_}
    '\d+.*externalsystemorderservice.*' {$col.add($Matches[0])}
    }

    $fields = $fields -replace '#Fields:'
    $fields = $fields.trim() -split ' '
    $col -replace ' ',',' | convertfrom-csv -Header $fields |
    Export-Csv -Path C:\temp\Test\final_dsfprdapp711.csv -NoTypeInformation
    }

    But, the date column is showing with latest log file date.

    Any suggestions?

    -Kalyan

  • #42910

    random commandline
    Participant

    Make this change. A csv is created for each log file.

    Export-Csv -Path C:\temp\Test\final_$($logFile.basename).csv -NoTypeInformation
    
  • #42953

    Richard Diphoorn
    Participant

    Do you want to have every single line in a separate csv file?

  • #42960

    Venkata Kalyan
    Participant

    Hi,
    I am planning to have all log files in one CSV file.
    But the headers column should be only one.

    -Kalyan

  • #43022

    Venkata Kalyan
    Participant

    Hi,
    To be more precise, requirement goes like the below:
    I have seven logfiles.
    > I need to go through each logfile and find a string (externalsystemorderservice) and get that line with limited headers (date,time,s-computername,s-ip,time-taken) to csv file.

    All logfiles data to a single CSV file.

    I am able to do for one logfile. But it comes to seven logfiles, I am not able to do.

    -Kalyan

  • #43059

    random commandline
    Participant

    What does your first matched line and fields look like? You will need to specify headers for the lines in your collection before exporting.
    This is an example.

    $col | ConvertFrom-Csv -Header 'cs','cs-host','sc-status','sc-bytes' -Delimiter ' ' |
    Export-Csv -Path C:\temp\Test\final_dsfprdapp711.csv -NoTypeInformation -Append
    
    # Results
    # cs                              cs-host                sc-status             sc-bytes             
    # --                                -------                ---------             --------             
    # https://domain.foo...  domain.foo           301                   581  
    
  • #43967

    Venkata Kalyan
    Participant

    Thanks. Let me try.

    -Kalyan

You must be logged in to reply to this topic.