Invoke-Webrequest AllElements not completing

Welcome Forums General PowerShell Q&A Invoke-Webrequest AllElements not completing

This topic contains 6 replies, has 4 voices, and was last updated by

 
Participant
1 year, 1 month ago.

  • Author
    Posts
  • #72826

    Participant
    Points: 0
    Rank: Member

    I must be doing something wrong. I have similar code but the script never finishes (even when I add where clauses, etc. ). Is it me or just a really big page? I let it run overnight and nothing.

    I'm trying to scrape specific info from Zillow.

    $URL = "https://www.zillow.com/homes/fsbo/house,condo,townhouse_type/2-_beds/150000-450000_price/552-1657_mp/30.702877,-85.093918,29.697596,-86.56334_rect/9_zm/df1c1593d8X1-CR1o9mzr7z62mim_ul9gt_crid/0_mmm/"
    
    # reading website data:
    $data = Invoke-WebRequest -Uri $URL
    $data.AllElements
    
    
  • #72829

    Keymaster
    Points: 1
    Rank: Member

    Shopping for houses?

    Which bit doesn't complete? The Invoke, or the attempt to access AllElements? Is this on a client or a server?

  • #72860

    Participant
    Points: 0
    Rank: Member

    Kind of. 🙂 My wife is in the Real Estate business and I made the mistake of saying, oh yeah, I can do that. It'll be easy with PowerShell.

    The attempt of AllElements and also the attempt at ParsedHtml... tried both.

    And it is on a laptop. Its a pretty fast laptop on wired network getting > 170 Mbit/s ... not too bad.

    • #72872

      Participant
      Points: 0
      Rank: Member

      I'm getting a similar result when I try to get info from that object. Not sure if it would help, but have you looked into getting API access? It might be easier to work with than screen scrapes.

    • #72874

      Participant
      Points: 0
      Rank: Member

      Yes, they charge an arm and a leg for API access and they limit how much you can pull back. Someone I know did this with Python and it worked great. I assumed it would be as easy (and a learning process) by doing it in PowerShell.

  • #73070

    Participant
    Points: 0
    Rank: Member

    So the lack of responses leads me to believe PowerShell is not the right choice for Web Scraping? Any ideas out there?

    • #79217

      Participant
      Points: 0
      Rank: Member

      This can be done. I would personally do this with some regex.
      What is it that you want to scrape? The images on the right side?

The topic ‘Invoke-Webrequest AllElements not completing’ is closed to new replies.