Log Parse Script

Tagged: 

This topic contains 7 replies, has 4 voices, and was last updated by Profile photo of Paul Paul 2 months, 3 weeks ago.

  • Author
    Posts
  • #65535
    Profile photo of Paul
    Paul
    Participant

    I have an example txt file with the below contents which I would like to ONLY extract it's substring. When I use the below command it's grabbing the entire line.

    #Log File location.
    $path = "C:\log.log"

    #Search through log for matching string.
    Get-Content $path | Select-String -Pattern "size","company","customerid"

    ======================================

    01-01-2012 01:13:36 Blah blah : blah CustomerId:1234 downloaded
    Blah Size:5432 bytes Carrier:Company-A
    01-01-2012 01:13:45 Blah blah : blah CustomerId:41444 downloaded
    Blah Size:38655 bytes Carrier:Company-S
    01-01-2012 01:13:47 Blah blah : blah CustomerId:12334 downloaded
    Blah Size:25632 bytes Carrier:Company-A
    01-01-2012 01:13:50 Blah blah : blah CustomerId:41444 downloaded
    Blah Size:7213 bytes Carrier:Company-S
    01-01-2012 01:13:58 Blah blah : blah CustomerId:553155 downloaded
    Blah Size:70100 bytes Carrier:Company-V

  • #65550
    Profile photo of Olaf Soyk
    Olaf Soyk
    Participant

    Are the log entries really 2 lines each? What is the expected result you want? Could you give an example with one line? If you post code or log file content you should use the code tags as explained above the post edit tool.

  • #65613
    Profile photo of random commandline
    random commandline
    Participant

    If each line starts with a date, this should work.

    $file = Get-ChildItem 'C:\log.log'
    $myobj = switch -Regex -File $file {
    'CustomerId:(.*?) .*size:(.*?) .*Carrier:(.*)$'
    {[PSCustomObject]@{ID=$Matches[1];Size=$Matches[2];Company=$Matches[3]}}
    }
    
    $group = $myobj | Group-Object -Property ID
    foreach ($g in $group){
        [PSCustomObject]@{
            ID = $g.Name
            TotalSize = ($g.group.size | Measure-Object -Sum).Sum
            Company = $g.group.Company | select -First 1
        }
    }
    # Results
    # ID     TotalSize Company  
    # 1234        5432 Company-A
    # 41444      45868 Company-S
    # 12334      25632 Company-A
    # 553155     70100 Company-V
    
  • #65626
    Profile photo of Sam Boutros
    Sam Boutros
    Participant

    try:

    #requires -Version 4
    
    $LogData = Get-Content .\log.log
    $Output = @()
    foreach ($Line in $LogData) {
        if ($Line -match 'CustomerId') {
            $NewRecord = $true
            $CustomerID = $Line.Substring($Line.LastIndexOf(':')+1,$Line.Length-$Line.LastIndexOf(':')-1).Split(' ')[0]
        } 
        if ($Line -match 'Size') {
            if ($NewRecord) {
                $Props = [ordered]@{
                    CustomerId = $CustomerID
                    Company    = $Line.Substring($Line.LastIndexOf(':')+1,$Line.Length-$Line.LastIndexOf(':')-1).Split(' ')[0]
                    Size       = $Line.Substring($Line.IndexOf(':')+1,$Line.Length-$Line.IndexOf(':')-1).Split(' ')[0]
                }
                $Output += New-Object -TypeName PSObject -Property $Props 
                $NewRecord = $false
            } # else this is an orphan, disregard
        }     
    }
    $Output | sort CustomerId | FT -a 
    

    You should get output like:

    CustomerId Company   Size 
    ---------- -------   ---- 
    12334      Company-A 25632
    1234       Company-A 5432 
    41444      Company-S 38655
    41444      Company-S 7213 
    553155     Company-V 70100
    
  • #65628
    Profile photo of Paul
    Paul
    Participant

    Thanks all! How would I total the size of each unique customer?

    • #65635
      Profile photo of Sam Boutros
      Sam Boutros
      Participant

      try:

      $Groups = $Output | group -Property CustomerID | Sort Count -Descending
      $Tally = @()
      foreach ($Item in $Groups) { 
          $Props = [ordered]@{
              CustomerId = $Item.Group.CustomerID | select -First 1
              Company    = $Item.Group.Company | select -First 1
              TotalSize  = $($Sum = 0; $Item.Group.Size | % { $Sum += $_ }; $Sum)
          }
          $Tally += New-Object -TypeName PSObject -Property $Props 
      }
      $Tally | sort CustomerId | FT -a 
      

      (This depends on $Output from above..)
      Example output:

      CustomerId Company   TotalSize
      ---------- -------   ---------
      12334      Company-A     31064
      41444      Company-S     45868
      553155     Company-V     70100
      
  • #65643
    Profile photo of random commandline
    random commandline
    Participant

    I edited my previous post to include total size for each customerid.

  • #65661
    Profile photo of Paul
    Paul
    Participant

    Thanks guys for the help.

You must be logged in to reply to this topic.