Invoke-RestMethod - 100 call Limit on an API.

This topic contains 3 replies, has 2 voices, and was last updated by  Dave Wyatt 2 years, 9 months ago.

  • Author
    Posts
  • #20478

    Lawrence Keyes
    Participant

    I'm processing lists with SmartyStreets, which validate snail mail addresses against the post office list of valid mailing addresses. (how quaint...I know...)

    I send this using two variables:
    $whole = the string http://api.smartystreets.com/street-address?" plus the user-id and user-authorization code
    $converted,= a reference to the list that was read in from a .CSV file using convertfrom-csv and convertto-json

    Invoke-RestMethod -Uri $whole -Body $converted -ContentType application/json -Method Post
    

    I"m able to insert an entire list with a single variable..($converted) .but if the list has over 100 addresses, I get an error back from the SmartyStreets API which says that the number of addresses that can be passed via a single call is 100.

    So, I"m looking for a simple way to chunk the call into groups of 100 addresses.

    Two ideas that I"ve considered:
    1. I could create a call for each address with a foreach loop...but that would create a lot of network traffic, and require the server to respond individually for each address.
    2. I could split up the larger .csv file into smaller files of 100 lines each, then make the call for each group of 100.

    Any ideas would be appreciated.

  • #20480

    Dave Wyatt
    Moderator

    I like to abuse Group-Object for this, personally. 🙂 In this example, $bigList is a simple array of integers, but this could also be the objects returned by your Import-Csv command on the large file:

    $bigList = 1..1000
    
    $counter = @{ Value = 0 }
    $groupSize = 100
    
    $groups = $bigList | Group-Object -Property { [math]::Floor($counter.Value++ / $groupSize) }
    
    foreach ($group in $groups)
    {
        $littleList = $group.Group
        # $littleList.Count will be somewhere between 1 and 100.  (The final group may have less than 100.)
    
        # Generate your $converted string here and call Invoke-RestMethod.
    }
    
    
  • #20482

    Lawrence Keyes
    Participant

    Dave...thanks so much. This looks great, and was exactly what I'm looking for. — L

  • #20484

    Dave Wyatt
    Moderator

    No problem! 🙂

You must be logged in to reply to this topic.