Invoke-RestMethod - 100 call Limit on an API.

This topic contains 4 replies, has 3 voices, and was last updated by  indhu 1 day, 13 hours ago.

  • Author
    Posts
  • #20478

    Lawrence Keyes
    Participant

    I'm processing lists with SmartyStreets, which validate snail mail addresses against the post office list of valid mailing addresses. (how quaint...I know...)

    I send this using two variables:
    $whole = the string http://api.smartystreets.com/street-address?" plus the user-id and user-authorization code
    $converted,= a reference to the list that was read in from a .CSV file using convertfrom-csv and convertto-json

    Invoke-RestMethod -Uri $whole -Body $converted -ContentType application/json -Method Post
    

    I"m able to insert an entire list with a single variable..($converted) .but if the list has over 100 addresses, I get an error back from the SmartyStreets API which says that the number of addresses that can be passed via a single call is 100.

    So, I"m looking for a simple way to chunk the call into groups of 100 addresses.

    Two ideas that I"ve considered:
    1. I could create a call for each address with a foreach loop...but that would create a lot of network traffic, and require the server to respond individually for each address.
    2. I could split up the larger .csv file into smaller files of 100 lines each, then make the call for each group of 100.

    Any ideas would be appreciated.

  • #20480

    Dave Wyatt
    Moderator

    I like to abuse Group-Object for this, personally. 🙂 In this example, $bigList is a simple array of integers, but this could also be the objects returned by your Import-Csv command on the large file:

    $bigList = 1..1000
    
    $counter = @{ Value = 0 }
    $groupSize = 100
    
    $groups = $bigList | Group-Object -Property { [math]::Floor($counter.Value++ / $groupSize) }
    
    foreach ($group in $groups)
    {
        $littleList = $group.Group
        # $littleList.Count will be somewhere between 1 and 100.  (The final group may have less than 100.)
    
        # Generate your $converted string here and call Invoke-RestMethod.
    }
    
    
  • #20482

    Lawrence Keyes
    Participant

    Dave...thanks so much. This looks great, and was exactly what I'm looking for. — L

  • #20484

    Dave Wyatt
    Moderator

    No problem! 🙂

  • #84736

    indhu
    Participant

    I still have this issue and couldn't figure out can anybody help? I am getting an API device and trying to insert the data to table. I am using Rest invoke method where the API returns only 100 devices.

    $Url= "https://api-blahblah /devices"
    #To load data from devices
    $Response = Invoke-RestMethod -Method Get -Uri $Url -Header $Headers
    $Devices=@()
    $Url= "https://api-blahblah /devices"
    $Response = Invoke-RestMethod -Method Get -Uri $Url -Header $Headers
    $Output = $OutputInterim.results | Select name, nickName
    $Devices += $Output

    Now manually we enter the cursor value and try to create 2000 records. Is there any possibility to fetch all record in a single call?

You must be logged in to reply to this topic.