Invoke-RestMethod - 100 call Limit on an API.

This topic contains 3 replies, has 2 voices, and was last updated by Profile photo of Dave Wyatt Dave Wyatt 2 years ago.

  • Author
    Posts
  • #20478
    Profile photo of Lawrence Keyes
    Lawrence Keyes
    Participant

    I'm processing lists with SmartyStreets, which validate snail mail addresses against the post office list of valid mailing addresses. (how quaint...I know...)

    I send this using two variables:
    $whole = the string http://api.smartystreets.com/street-address?" plus the user-id and user-authorization code
    $converted,= a reference to the list that was read in from a .CSV file using convertfrom-csv and convertto-json

    Invoke-RestMethod -Uri $whole -Body $converted -ContentType application/json -Method Post
    

    I"m able to insert an entire list with a single variable..($converted) .but if the list has over 100 addresses, I get an error back from the SmartyStreets API which says that the number of addresses that can be passed via a single call is 100.

    So, I"m looking for a simple way to chunk the call into groups of 100 addresses.

    Two ideas that I"ve considered:
    1. I could create a call for each address with a foreach loop...but that would create a lot of network traffic, and require the server to respond individually for each address.
    2. I could split up the larger .csv file into smaller files of 100 lines each, then make the call for each group of 100.

    Any ideas would be appreciated.

  • #20480
    Profile photo of Dave Wyatt
    Dave Wyatt
    Moderator

    I like to abuse Group-Object for this, personally. 🙂 In this example, $bigList is a simple array of integers, but this could also be the objects returned by your Import-Csv command on the large file:

    $bigList = 1..1000
    
    $counter = @{ Value = 0 }
    $groupSize = 100
    
    $groups = $bigList | Group-Object -Property { [math]::Floor($counter.Value++ / $groupSize) }
    
    foreach ($group in $groups)
    {
        $littleList = $group.Group
        # $littleList.Count will be somewhere between 1 and 100.  (The final group may have less than 100.)
    
        # Generate your $converted string here and call Invoke-RestMethod.
    }
    
    
  • #20482
    Profile photo of Lawrence Keyes
    Lawrence Keyes
    Participant

    Dave...thanks so much. This looks great, and was exactly what I'm looking for. — L

  • #20484
    Profile photo of Dave Wyatt
    Dave Wyatt
    Moderator

    No problem! 🙂

You must be logged in to reply to this topic.