Parse Log File And Put Some Info Into CSV

This topic contains 5 replies, has 3 voices, and was last updated by Profile photo of Chris Cass Chris Cass 2 days ago.

  • Author
    Posts
  • #69060
    Profile photo of Chris Cass
    Chris Cass
    Participant

    Another newbie question...

    I have a log file for an application that runs for an entire month, tracking how many people are logged into an application at a given time. It runs hourly in production environments (though the example I'm about to paste is from a test environment, so it's running much less often). The output in the file looks like so:

    LicenseTrace Verbose;1;2017-04-05 14:34;Initiating Application
    LicenseTrace Verbose;17;2017-04-05 15:00;system.admin 1.0;0;system.maxusers 1.0;0;system.maxusers 2.0;0;system.base 1.0;1;system.ipod 1.0;0;system.blackberry 1.0;0
    LicenseTrace Verbose;18;2017-04-05 15:02;system.admin 1.0;0;system.maxusers 1.0;0;system.maxusers 2.0;0;system.base 1.0;1;system.ipod 1.0;0;system.blackberry 1.0;0
    LicenseTrace Verbose;19;2017-04-05 15:02;Ending Application
    LicenseTrace Verbose;1;2017-04-05 15:04;Initiating Application
    LicenseTrace Verbose;3;2017-04-05 15:27;system.admin 1.0;0;system.maxusers 1.0;0;system.maxusers 2.0;0;system.base 1.0;0;system.ipod 1.0;0;system.blackberry 1.0;0
    LicenseTrace Verbose;4;2017-04-05 15:27;Ending Application
    LicenseTrace Verbose;1;2017-04-10 08:38;Initiating Application
    LicenseTrace Verbose;3;2017-04-10 08:40;system.admin 1.0;0;system.maxusers 1.0;0;system.maxusers 2.0;0;system.base 1.0;1;system.ipod 1.0;0;system.blackberry 1.0;0
    LicenseTrace Verbose;4;2017-04-10 08:40;Ending Application
    LicenseTrace Verbose;1;2017-04-10 08:42;Initiating Application
    LicenseTrace Verbose;9;2017-04-10 09:00;system.admin 1.0;0;system.maxusers 1.0;0;system.maxusers 2.0;0;system.base 1.0;2;system.ipod 1.0;0;system.blackberry 1.0;0
    LicenseTrace Verbose;10;2017-04-10 09:08;system.admin 1.0;0;system.maxusers 1.0;0;system.maxusers 2.0;0;system.base 1.0;2;system.ipod 1.0;0;system.blackberry 1.0;0
    LicenseTrace Verbose;11;2017-04-10 09:08;Ending Application
    LicenseTrace Verbose;1;2017-04-10 09:13;Initiating Application
    LicenseTrace Verbose;3;2017-04-10 09:35;system.admin 1.0;0;system.maxusers 1.0;0;system.maxusers 2.0;0;system.base 1.0;0;system.ipod 1.0;0;system.blackberry 1.0;0
    LicenseTrace Verbose;4;2017-04-10 09:35;Ending Application
    LicenseTrace Verbose;1;2017-04-13 15:23;Initiating Application
    LicenseTrace Verbose;3;2017-04-13 15:48;system.admin 1.0;0;system.maxusers 1.0;0;system.maxusers 2.0;0;system.base 1.0;1;system.ipod 1.0;0;system.blackberry 1.0;0
    LicenseTrace Verbose;4;2017-04-13 15:48;Ending Application
    

    My end goal is to create a .csv file listing the date/time in one column and the number to the right of "system.base 1.0;" for every row in the log.

    So far in my pursuit I have figured out how to hit the registry to get the location of the application configuration files, search the specific configuration file that tells me the location of the log, create variables to come up with the name of the current log file (for this example we'll call it license-2017-04.log). I've used all of that to create a variable called $currentlog that gets me to the location and file of the current log. I've run a get-content to verify I can pull up the contents of the log file based on all of those parameters (though I'm pretty sure the get-content will not be part of my final script, it just allowed me to verify I was hitting the right place based on all the variables). But now I'm stuck...

    I've seen all sorts of examples, but they all seem to revolve around the file having the same contents on each line... So I know I need to parse out just the lines -like "*system.maxusers*" and then somehow separate each line to display just the date/time and the base users... Any suggestions?

  • #69073
    Profile photo of Olaf Soyk
    Olaf Soyk
    Participant

    usually we expect a little more efford from you ... and that you post at least a little bit of the code you already created ... 😉
    That could be a starting point for you:

    $LogFileContent = Get-Content -Path 'Path to your logfile'
    Foreach($line in $LogFileContent){
        if($line -match 'Verbose\;\d+\;(\d{4}(-\d{2}){2}\s+\d{2}:\d{2})\;system.*system\.base\s+1\.0;(\d+)\;system\.'){
            [PSCustomObject]@{
                'DateTime' = Get-Date $Matches[1]
                'Count' = [INT]$Matches[3]
            }
        }
    }
  • #69123
    Profile photo of random commandline
    random commandline
    Participant

    If this is a large log file, you may want to use a switch statement.

    $file = Get-ChildItem '.\path\to\file.log'
    
    $results = switch -Regex -File $file {
        ';(\d{4}.*);system.admin.*system.base (\d.*?);system' 
        {[PSCustomObject]@{Date = $Matches[1] ; SystemBase = $Matches[2]}}
    }
    
    $results | Export-Csv '.\path\to\file.csv' -NoTypeInformation
    
  • #69166
    Profile photo of Chris Cass
    Chris Cass
    Participant

    Thank you both. Sorry for not posting any attempted code. Literally nothing I saw made sense. Case in point, I have no idea what the third line in Olaf's code, or the fourth line in RC's code is doing... But Olaf's code worked. My project is now complete, so thank you both. Now I need to educate myself on what that line is doing.

    • #69177
      Profile photo of Olaf Soyk
      Olaf Soyk
      Participant

      😉 Have you ever heard about regular expressions? Mine is just a little more complex than rc's but it's doing almost the same.

    • #69180
      Profile photo of Chris Cass
      Chris Cass
      Participant

      Started reading about them this morning. Thanks!

You must be logged in to reply to this topic.