log content to json format

This topic contains 4 replies, has 3 voices, and was last updated by  Rob Simmers 6 days, 16 hours ago.

  • Author
    Posts
  • #102382

    Vishal Joshi
    Participant

    I am trying to get a content from log file and convert it into a json format. for example below is log file content

    2018-06-11 16:13:32,445 application="abc" category="msgevent" messagingEventType="msgreceived" messageType="bpl" barcode="323210590211613310641030"
    2018-06-11 16:13:33,097 application="abc" category="msgevent" messagingEventType="msgreceived" messageType="tpi" barcode="323210590211613310641030"
    2018-06-11 16:13:34,210 application="abc" category="msgevent" messagingEventType="msgreceived" messageType="clf" barcode="323210590211613330642030"

    i an trying to get json format like below for each line
    [{
    "timestamp": "2018-06-11 16:13:32",
    "application":"abc",
    "category":"msgevent",
    "messagingEventType":"msgreceived",
    "messageType":"bpl",
    "barcode":"323210590211613310641030"

    }]

    So i have written a code
    $abc = gci -path monitoring.log | select-string -pattern "*application*" | out-string

    $abc.split(' ')
    $abc -replace '=' ,':' | ConvertTo-Json

    but it is not working.

  • #102383

    Fredrik Kacsmarck
    Participant

    Well there are several issues.

    1. You don't store the result from the split anywhere.
    Just running $abc.split(' ') will output the results but doesn't perform and store the split results in $abc automagically.

    2. You're splitting on space, which mean that the date and time will be in two seperate cells in the array.
    So you need to write some code to combine those again etc.

    3. Do the replace before the split, otherwise you would need to do it via e.g. a foreach loop to loop through the results from the split and then replace the character.

    Then it depends on if ConvertTo-Json will be able to parse it straight away or if you need to create the array or hashtable yourself before doing the convert.
    So you may need to do additional work to get it to accept your input.

  • #102410

    Rob Simmers
    Participant

    Your best bet is to parse it to a PSObject then convert it or export it in the output you want.

    $log = @"
    2018-06-11 16:13:32,445 application="abc" category="msgevent" messagingEventType="msgreceived" messageType="bpl" barcode="323210590211613310641030"
    2018-06-11 16:13:33,097 application="abc" category="msgevent" messagingEventType="msgreceived" messageType="tpi" barcode="323210590211613310641030"
    2018-06-11 16:13:34,210 application="abc" category="msgevent" messagingEventType="msgreceived" messageType="clf" barcode="323210590211613330642030"
    "@
    
    #Emulate Get-Content
    $log = $log.Split([environment]::NewLine) | Where{$_}
    
    $results = foreach ($line in $log) {
        $arr = $line.Split(" ")
    
        New-Object -TypeName PSObject -Property @{
            DateTime           = ("{0} {1}" -f $arr[0], $arr[1]).Split(",")[0]
            Application        = $arr[2].Split("=")[1].Replace('"','')
            Category           = $arr[3].Split("=")[1].Replace('"','')
            MessagingEventType = $arr[4].Split("=")[1].Replace('"','')
            MessageType        = $arr[5].Split("=")[1].Replace('"','')
            BarCode            = $arr[6].Split("=")[1].Replace('"','')
        }
    }
    
    $results | ConvertTo-Json
    

    Output:

    [
        {
            "MessageType":  "bpl",
            "MessagingEventType":  "msgreceived",
            "BarCode":  "323210590211613310641030",
            "Application":  "abc",
            "Category":  "msgevent",
            "DateTime":  "2018-06-11 16:13:32"
        },
        {
            "MessageType":  "tpi",
            "MessagingEventType":  "msgreceived",
            "BarCode":  "323210590211613310641030",
            "Application":  "abc",
            "Category":  "msgevent",
            "DateTime":  "2018-06-11 16:13:33"
        },
        {
            "MessageType":  "clf",
            "MessagingEventType":  "msgreceived",
            "BarCode":  "323210590211613330642030",
            "Application":  "abc",
            "Category":  "msgevent",
            "DateTime":  "2018-06-11 16:13:34"
        }
    ]
    
    
    • #102439

      Vishal Joshi
      Participant

      Thanks This is work for me bu one issue i was trying to make it dynamically means if any new element get added in logs then it will auto add in json. don't want to define static as below also see the logs it will get added new content then it should be add that in json

      New-Object -TypeName PSObject -Property @{
      DateTime = ("{0} {1}" -f $arr[0], $arr[1]).Split(",")[0]
      Application = $arr[2].Split("=")[1].Replace('"',")
      Category = $arr[3].Split("=")[1].Replace('"',")
      MessagingEventType = $arr[4].Split("=")[1].Replace('"',")
      MessageType = $arr[5].Split("=")[1].Replace('"',")
      BarCode = $arr[6].Split("=")[1].Replace('"',")
      }
      }

      $log = @"
      2018-06-11 16:13:32,445 application="abc" category="msgevent" messagingEventType="msgreceived" messageType="bpl" barcode="323210590211613310641030" abc="pdf"
      2018-06-11 16:13:33,097 application="abc" category="msgevent" messagingEventType="msgreceived" messageType="tpi" barcode="323210590211613310641030" bedf="asdad"
      2018-06-11 16:13:34,210 application="abc" category="msgevent" messagingEventType="msgreceived" messageType="clf" barcode="323210590211613330642030"
      "@

  • #102472

    Rob Simmers
    Participant

    You would use Get-Content to get the logs. The static example was to test the parsing, note the comment that it is emulating Get-Content.

    $log = Get-Content -Path C:\MyLogs\Log123.log
    
    $results = foreach ($line in $log) {...
    

    Get-Content will return an array of lines

You must be logged in to reply to this topic.