Invoke-Webrequest AllElements not completing

This topic contains 5 replies, has 3 voices, and was last updated by Profile photo of Mike Lawell Mike Lawell 1 week ago.

  • Author
    Posts
  • #72826
    Profile photo of Mike Lawell
    Mike Lawell
    Participant

    I must be doing something wrong. I have similar code but the script never finishes (even when I add where clauses, etc. ). Is it me or just a really big page? I let it run overnight and nothing.

    I'm trying to scrape specific info from Zillow.

    $URL = "https://www.zillow.com/homes/fsbo/house,condo,townhouse_type/2-_beds/150000-450000_price/552-1657_mp/30.702877,-85.093918,29.697596,-86.56334_rect/9_zm/df1c1593d8X1-CR1o9mzr7z62mim_ul9gt_crid/0_mmm/"
    
    # reading website data:
    $data = Invoke-WebRequest -Uri $URL
    $data.AllElements
    
    
  • #72829
    Profile photo of Don Jones
    Don Jones
    Keymaster

    Shopping for houses?

    Which bit doesn't complete? The Invoke, or the attempt to access AllElements? Is this on a client or a server?

  • #72860
    Profile photo of Mike Lawell
    Mike Lawell
    Participant

    Kind of. 🙂 My wife is in the Real Estate business and I made the mistake of saying, oh yeah, I can do that. It'll be easy with PowerShell.

    The attempt of AllElements and also the attempt at ParsedHtml... tried both.

    And it is on a laptop. Its a pretty fast laptop on wired network getting > 170 Mbit/s ... not too bad.

    • #72872
      Profile photo of Colin Nichols
      Colin Nichols
      Participant

      I'm getting a similar result when I try to get info from that object. Not sure if it would help, but have you looked into getting API access? It might be easier to work with than screen scrapes.

    • #72874
      Profile photo of Mike Lawell
      Mike Lawell
      Participant

      Yes, they charge an arm and a leg for API access and they limit how much you can pull back. Someone I know did this with Python and it worked great. I assumed it would be as easy (and a learning process) by doing it in PowerShell.

  • #73070
    Profile photo of Mike Lawell
    Mike Lawell
    Participant

    So the lack of responses leads me to believe PowerShell is not the right choice for Web Scraping? Any ideas out there?

You must be logged in to reply to this topic.