Invoke-WebRequest does not display classname

This topic contains 2 replies, has 3 voices, and was last updated by  Mike Lawell 6 months ago.

  • Author
    Posts
  • #68757

    random commandline
    Participant

    I am looking for a specific classname under the element with an id of 'control'. The New-Object method displays the classname, but Invoke-WebRequest does not. Have you encountered sites that do this or am I overlooking something?

    # New-Object Method
    $ie = New-Object -com "InternetExplorer.Application"
    $ie.Visible = $true
    $ie.Navigate($url)
    while ($ie.Busy){Start-Sleep 1}
    $doc = $ie.Document
    $control = $doc.getElementById('control')
    $href = $control.childNodes | Where-Object {
        $_.classname -eq 'specific classname'} |
        Select-Object href 
    
    # Invoke Method
    $test = Invoke-WebRequest -Uri $url 
    $controli = $test.ParsedHtml.getElementById('control') 
    $href = $controli.childNodes  | Where-Object {
        $_.classname -eq 'specific classname'} |
        Select-Object href
    
  • #68760

    Bryce McDonald
    Participant

    With Invoke-Webrequest what we may need to do is first select all elements, then narrow our results. For instance:

    $test = Invoke-WebRequest -Uri $url
    $results = $test.AllElements | Where Class -eq "classname" | Where ID -eq "control" | Select-Object href
    

    This works in my testing, but it may be a little different in your environment. Let me know!

  • #72781

    Mike Lawell
    Participant

    I must be doing something wrong. I have similar code but the script never finishes (even when I add where clauses, etc. ). Is it me or just a really big page? I let it run overnight and nothing.

    I'm trying to scrape specific info from Zillow.

    $URL = "https://www.zillow.com/homes/fsbo/house,condo,townhouse_type/2-_beds/150000-450000_price/552-1657_mp/30.702877,-85.093918,29.697596,-86.56334_rect/9_zm/df1c1593d8X1-CR1o9mzr7z62mim_ul9gt_crid/0_mmm/"
    
    # reading website data:
    $data = Invoke-WebRequest -Uri $URL
    $data.AllElements
    

You must be logged in to reply to this topic.