Downloading an HTTP table to an array and insert to PRTG

Welcome Forums General PowerShell Q&A Downloading an HTTP table to an array and insert to PRTG

This topic contains 2 replies, has 2 voices, and was last updated by

 
Participant
1 month ago.

  • Author
    Posts
  • #121854

    Participant
    Points: 8
    Rank: Member

    Downloading an HTTP table to an array
    https://community.spiceworks.com/topic/543224-downloading-an-http-table-to-an-array

    HI all, I would be very grateful for your help with a script I based upon the above article.

    I have managed to get some parts of it it to work, extracting information from
    innerHTML, innerText, outerHTML and outerText (below) but
    I have trouble with presenting results and feeding this into a variable so result can be uploaded to the PRTG monitoring server.

    Much appreciating your help,

    Thanks!

    $ie = New-Object -ComObject "InternetExplorer.Application"
    #$ie.visible = $true
    $ie.silent = $true
    $ie.Navigate("http://download.webpage.com/webpage/test.html")
    while($ie.ReadyState -ne 4) {start-sleep -m 100}
    #while ($ie.Busy) {Start-Sleep -Milliseconds 50}
    
    Write-Warning "Retrieving information from web"
    $body = $ie.Document.getElementsByTagName("table")
    
    Write-Warning "Tables : "
    $tables = $body
    $tables
    
    $tables2 = $tables | select innerText
    Write-Warning "Tables 2: "
    $tables2
    
    $RawTables = $body
    
    Write-Verbose "Found $($RawTables.Count) tables in HTML"
    $TableNum = 1
    
    ForEach ($RawTable in $RawTables)
    { Write-Verbose "Parsing headers for Table #$TableNum"
    
    $InnerHTML = ($RawTable | select innerText).innerText -split "`n"
    
    Write-Warning "InnerHTML: "
    $InnerHTML
    
    $Search = $InnerHTML
    Write-Warning "Search: "
    
    $Search
    
    $Fields = [ordered]@{}
    ForEach ($Line in $Search)
    { ForEach ($Field in $Line.Matches)
    { $Fields.Add($Field.Groups[1].Value,$null)
    }
    }
    
    Write-Warning "Parsing table data for Table #$TableNum"
    If ($Fields.Count -gt 0)
    
    { $Search = $InnerHTML | Select-String -Pattern "(.*?)" -AllMatches
    
    Write-Warning "Search2: "
    $Search
    $Row = [PSCustomObject]$Fields
    $Index = 0
    $Table = ForEach ($Match in $Search.Matches)
    { $Field = $Fields.GetEnumerator().Name[$Index]
    $Row.$Field = ($Match.Groups[1].Value -replace "< .*?>","").Trim()
    If ($Index -eq ($Fields.Count - 1))
    { $Index = 0
    Write-Output $Row
    $Row = [PSCustomObject]$Fields
    }
    Else
    { $Index ++
    }
    }
    [PSCustomObject]@{
    Raw = $InnerHTML
    Data = $Table
    Fields = $Fields
    }
    }
    Else
    { Write-Warning "Unable to determine headers for Table #$TableNum, skipping it"
    }
    $TableNum ++
    }

    # This code will need to be inserted as well (not finished)
    # Sends results to PRTG for monitoring:

    #""
    #
    #     ""
    #     "Javascript enabled"
    #     "$yesno"
    #     "
    "

    #     ""
    #     "Client IP address"
    #     "$clientip"
    #     "
    "
    #     "$returndescription"

    #     ""
    #     "Real connecting IP address"
    #     "$realip"
    #     "
    "

    #     ""
    #     "Server IP address"
    #     "$serverip"
    #     "
    "

    #     "$returndescription"

    #     ""
    #     "M_IP1"
    #     "$m_ip1"
    #     "
    "

    #     ""
    #     "M_IP2"
    #     "$m_ip2"
    #     "
    "

    #     "$returndescription"

    #     ""
    #     "M_IP1"
    #     "$m_ip1"
    #     "
    "

    #     ""
    #     "IP mapp DNS mapp"
    #     "$imdm"
    #     "
    "

    #     "$returndescription"

    #     ""
    #     "Serial"
    #     "$serial"
    #     "
    "

    #     ""
    #     "RTT Small hreq"
    #     "$rttsmall"
    #     "
    "

    #     ""
    #     "BW DL Small"
    #     "$bwdlsmalldl"
    #     "
    "

    #     ""
    #     "BW UL Med"
    #     "$bwmedul"
    #     "
    "

    #     ""
    #     "BW DL Large"
    #     "$bwlargedl"
    #     "
    "

    #     ""
    #     "BW UL Large"
    #     "$bwdllargeul"
    #     "
    "

    #""

    [System.Runtime.Interopservices.Marshal]::ReleaseComObject($ie)
    Remove-Variable ie

    # End of Script
    ================================================================
    This is the innerHTML, innerText, outerHTML and outerText I retrieve the information out from.

    innerHTML                   :

    JavaScript enabledYes
    Client IP address11.11.11.111Real connecting IP address11.11.11.111Server IP address33.333.333.33 (machine ip = 44.444.444.44)M_IP1:66.666.666.66M_IP2:55.555.555.55IP mapping matches DNS mapping:NoNetwork name:Network NameSerial:1234RTT for small http request17 msBW for DL small kbyte to browser55183.6 kbps   (HIT)BW for UL medium kbyte from browser29681.1 kbpsBW for DL large Mbyte to browserCalculating...BW for UL Large kbyte from browserCalculating...

    innerText                   :
    JavaScript enabled
    document.write('Yes'); Yes No

    Client IP address
    11.11.11.111

    Real connecting IP address
    11.11.11.111

    Server IP address
    33.333.333.33 (machine ip = 44.444.444.44)

    M_IP1:
    66.666.666.66

    M_IP2:
    55.555.555.55

    IP mapping matches DNS mapping:
    No

    Network name:
    Network Name

    Serial:
    1234

    RTT for small http request
    17 ms

    BW for download small kbyte to browser
    1001.1 kbps (HIT)

    BW for UL medium kbyte from browser
    1001.1 kbps

    BW for DL large Mbyte to browser
    Calculating...

    BW for UL Large kbyte from browser
    Calculating...
    outerHTML                   :

    JavaScript enabled Yes
    Client IP address 11.11.11.111
    Real connecting IP address 11.11.11.111
    Server IP address 33.333.333.33 (machine ip = 44.444.444.44)
    M_IP1: 66.666.666.66
    M_IP2: 55.555.555.55
    IP mapping matches DNS mapping: No
    Network name: Network Name
    Serial: 1234
    RTT for small http request 17 ms
    BW for DL small kbyte to browser 1001.1 kbps   (HIT)
    BW for UL medium kbyte from browser 1001.1 kbps
    BW for DL large Mbyte to browser Calculating...
    BW for UL Large kbyte from browser Calculating...

    outerText                   :
    JavaScript enabled
    document.write('Yes'); Yes No

    Client IP address
    11.11.11.111

    Real connecting IP address
    11.11.11.111

    Server IP address
    33.333.333.33 (machine ip = 44.444.444.44)

    M_IP1:
    66.666.666.66

    M_IP2:
    55.555.555.55

    IP mapping matches DNS mapping:
    No

    Network name:
    Network Name

    Serial:
    1234

    RTT for small http request
    17 ms

    BW for DL small kbyte to browser
    1001.1 kbps (HIT)

    BW for UL medium kbyte from browser
    1001.1 kbps

    BW for DL large Mbyte to browser
    Calculating...

    BW for UL Large kbyte from browser
    Calculating...

  • #121864

    Moderator
    Points: 98
    Team MemberHelping Hand
    Rank: Member

    Hey! Welcome!

    First up, can I recommend you chuck all of the code within that post into a [`pre`] block (see the notes above the editor). It's quite hard to read, as is.

    Secondly, (and this might be an issue with the formatting, as above, where I've not seen the question for the code) – what are you actually trying to achieve here? You say you're having difficulty formatting the output?

  • #121920

    Participant
    Points: 8
    Rank: Member

    Hi again,  thank you for your message and help!

    I have trouble with presenting results and feeding this into a variable so result can be uploaded to the PRTG monitoring server.
    The error I get when running code is:
    "
    WARNING: Parsing table data for Table #1
    WARNING: Unable to determine headers for Table #1, skipping it
    "

    It seems I can extract the lines with this code:

    $InnerHTML = ($RawTable | select innerText).innerText -split "`n"
    
    

    But then when ordering ($Fields) I get the error message since the IF clause doesnt work and ELSE clause kicks in:

    $Fields = [ordered]@{}
    ForEach ($Line in $Search)
    { ForEach ($Field in $Line.Matches)
    { $Fields.Add($Field.Groups[1].Value,$null)
    }
    }
    
    Write-Warning "Parsing table data for Table #$TableNum"
    If ($Fields.Count -gt 0)
    
    { $Search = $InnerHTML | Select-String -Pattern "(.*?)" -AllMatches
    
    Write-Warning "Search2: "
    $Search
    $Row = [PSCustomObject]$Fields
    $Index = 0
    $Table = ForEach ($Match in $Search.Matches)
    { $Field = $Fields.GetEnumerator().Name[$Index]
    $Row.$Field = ($Match.Groups[1].Value -replace "< .*?>","").Trim()
    If ($Index -eq ($Fields.Count – 1))
    { $Index = 0
    Write-Output $Row
    $Row = [PSCustomObject]$Fields
    }
    Else { $Index ++
    }
    }
    [PSCustomObject]@{
    Raw = $InnerHTML
    Data = $Table
    Fields = $Fields
    }
    }
    Else
    { Write-Warning "Unable to determine headers for Table #$TableNum, skipping it"
    }

    I need also to correct

    $Row.$Field = ($Match.Groups[1].Value -replace "< .*?>","").Trim()
    

    but that is probably the next issue when I get the IF clause to work.

     

You must be logged in to reply to this topic.