Invoke-Webrequest AllElements not completing

This topic contains 5 replies, has 3 voices, and was last updated by  Mike Lawell 2 months ago.

  • Author
    Posts
  • #72826

    Mike Lawell
    Participant

    I must be doing something wrong. I have similar code but the script never finishes (even when I add where clauses, etc. ). Is it me or just a really big page? I let it run overnight and nothing.

    I'm trying to scrape specific info from Zillow.

    $URL = "https://www.zillow.com/homes/fsbo/house,condo,townhouse_type/2-_beds/150000-450000_price/552-1657_mp/30.702877,-85.093918,29.697596,-86.56334_rect/9_zm/df1c1593d8X1-CR1o9mzr7z62mim_ul9gt_crid/0_mmm/"
    
    # reading website data:
    $data = Invoke-WebRequest -Uri $URL
    $data.AllElements
    
    
  • #72829

    Don Jones
    Keymaster

    Shopping for houses?

    Which bit doesn't complete? The Invoke, or the attempt to access AllElements? Is this on a client or a server?

  • #72860

    Mike Lawell
    Participant

    Kind of. 🙂 My wife is in the Real Estate business and I made the mistake of saying, oh yeah, I can do that. It'll be easy with PowerShell.

    The attempt of AllElements and also the attempt at ParsedHtml... tried both.

    And it is on a laptop. Its a pretty fast laptop on wired network getting > 170 Mbit/s ... not too bad.

    • #72872

      Colin Nichols
      Participant

      I'm getting a similar result when I try to get info from that object. Not sure if it would help, but have you looked into getting API access? It might be easier to work with than screen scrapes.

    • #72874

      Mike Lawell
      Participant

      Yes, they charge an arm and a leg for API access and they limit how much you can pull back. Someone I know did this with Python and it worked great. I assumed it would be as easy (and a learning process) by doing it in PowerShell.

  • #73070

    Mike Lawell
    Participant

    So the lack of responses leads me to believe PowerShell is not the right choice for Web Scraping? Any ideas out there?

You must be logged in to reply to this topic.