Invoke-Webrequest AllElements not completing

This topic contains 6 replies, has 4 voices, and was last updated by  Snak3d0c 10 months, 2 weeks ago.

  • Author
  • #72826

    Mike Lawell

    I must be doing something wrong. I have similar code but the script never finishes (even when I add where clauses, etc. ). Is it me or just a really big page? I let it run overnight and nothing.

    I'm trying to scrape specific info from Zillow.

    $URL = ",condo,townhouse_type/2-_beds/150000-450000_price/552-1657_mp/30.702877,-85.093918,29.697596,-86.56334_rect/9_zm/df1c1593d8X1-CR1o9mzr7z62mim_ul9gt_crid/0_mmm/"
    # reading website data:
    $data = Invoke-WebRequest -Uri $URL
  • #72829

    Don Jones

    Shopping for houses?

    Which bit doesn't complete? The Invoke, or the attempt to access AllElements? Is this on a client or a server?

  • #72860

    Mike Lawell

    Kind of. 🙂 My wife is in the Real Estate business and I made the mistake of saying, oh yeah, I can do that. It'll be easy with PowerShell.

    The attempt of AllElements and also the attempt at ParsedHtml... tried both.

    And it is on a laptop. Its a pretty fast laptop on wired network getting > 170 Mbit/s ... not too bad.

    • #72872

      Colin Nichols

      I'm getting a similar result when I try to get info from that object. Not sure if it would help, but have you looked into getting API access? It might be easier to work with than screen scrapes.

    • #72874

      Mike Lawell

      Yes, they charge an arm and a leg for API access and they limit how much you can pull back. Someone I know did this with Python and it worked great. I assumed it would be as easy (and a learning process) by doing it in PowerShell.

  • #73070

    Mike Lawell

    So the lack of responses leads me to believe PowerShell is not the right choice for Web Scraping? Any ideas out there?

    • #79217


      This can be done. I would personally do this with some regex.
      What is it that you want to scrape? The images on the right side?

You must be logged in to reply to this topic.