Author Posts

November 10, 2014 at 9:54 am

I'm processing lists with SmartyStreets, which validate snail mail addresses against the post office list of valid mailing addresses. (how quaint...I know...)

I send this using two variables:
$whole = the string http://api.smartystreets.com/street-address?" plus the user-id and user-authorization code
$converted,= a reference to the list that was read in from a .CSV file using convertfrom-csv and convertto-json

Invoke-RestMethod -Uri $whole -Body $converted -ContentType application/json -Method Post

I"m able to insert an entire list with a single variable..($converted) .but if the list has over 100 addresses, I get an error back from the SmartyStreets API which says that the number of addresses that can be passed via a single call is 100.

So, I"m looking for a simple way to chunk the call into groups of 100 addresses.

Two ideas that I"ve considered:
1. I could create a call for each address with a foreach loop...but that would create a lot of network traffic, and require the server to respond individually for each address.
2. I could split up the larger .csv file into smaller files of 100 lines each, then make the call for each group of 100.

Any ideas would be appreciated.

November 10, 2014 at 10:07 am

I like to abuse Group-Object for this, personally. 🙂 In this example, $bigList is a simple array of integers, but this could also be the objects returned by your Import-Csv command on the large file:

$bigList = 1..1000

$counter = @{ Value = 0 }
$groupSize = 100

$groups = $bigList | Group-Object -Property { [math]::Floor($counter.Value++ / $groupSize) }

foreach ($group in $groups)
{
    $littleList = $group.Group
    # $littleList.Count will be somewhere between 1 and 100.  (The final group may have less than 100.)

    # Generate your $converted string here and call Invoke-RestMethod.
}

November 10, 2014 at 10:18 am

Dave...thanks so much. This looks great, and was exactly what I'm looking for. — L

November 22, 2017 at 1:58 am

I still have this issue and couldn't figure out can anybody help? I am getting an API device and trying to insert the data to table. I am using Rest invoke method where the API returns only 100 devices.

$Url= "https://api-blahblah /devices"
#To load data from devices
$Response = Invoke-RestMethod -Method Get -Uri $Url -Header $Headers
$Devices=@()
$Url= "https://api-blahblah /devices"
$Response = Invoke-RestMethod -Method Get -Uri $Url -Header $Headers
$Output = $OutputInterim.results | Select name, nickName
$Devices += $Output

Now manually we enter the cursor value and try to create 2000 records. Is there any possibility to fetch all record in a single call?