Web Scraping with Invoke-WebRequest

Welcome Forums General PowerShell Q&A Web Scraping with Invoke-WebRequest

Viewing 2 reply threads
  • Author
    Posts
    • #257684
      Participant
      Topics: 1
      Replies: 1
      Points: 18
      Rank: Member

      Hi All,

      I am new to PowerShell. I was learning about web scraping I have created below script to fetch data from flipkart and save it to excel.

      I am facing 2 issues

      1st issue : whenever I start new PowerShell console window the script works fine for first 1 or 2 attempt after that it shows error like below:

      2nd issue : Every product has Name, Rating, Prize but other information may not available How to check if childNodes[] is present or not. it gives me error like this if any product has not any childNodes but script able to add all other available data

      Thanks

       

    • #257852
      Participant
      Topics: 16
      Replies: 1794
      Points: 3,300
      Helping Hand
      Rank: Community Hero

      Avoid Excel unless it’s required. It’s better to create a Powershell object and then export rather than parsing like that. Looking at the structure of the page, it’s a better approach to use names, ids and classnames to parse than all of the childnode indexes:

      Output:

      The parsing needs work as you the span can have additional populated information and there were some instances where the warranty was in the incorrect place. You can check the count of items and try to align them, but you would need to understand all the possible options in the SPAN. Also received the error after running it numerous times, not sure what is going on there as I attempted to clear variables but it still produced the error. Hopefully the above will make things easier for you.

    • #258302
      Participant
      Topics: 1
      Replies: 1
      Points: 18
      Rank: Member

      Thanks for replay. I really appreciate your help. I will learn and will follow your advice, thanks.

Viewing 2 reply threads
  • You must be logged in to reply to this topic.