API Rate Limit to 100 or Expect100Continue Question

Welcome Forums General PowerShell Q&A API Rate Limit to 100 or Expect100Continue Question

Viewing 5 reply threads
  • Author
    Posts
    • #203441
      Participant
      Topics: 1
      Replies: 2
      Points: 19
      Rank: Member

      I am looking for assistance  for one of powershell command that is running in foreach loop — for @ 150 times.

      Below command runs fine for 100 times and then start throwing error.  it looks like it is client side-Powershell side error as our API vendor said, it is some sort of limit on powershell.

      Command : Invoke-RestMethod -Uri https://SEPSERVER:8446/sepm/api/v1/computers?computerName=$SEPComputerMove -Headers $header -Method GET |select -Expandproperty numberOfElements -Debug

      Error

      Invoke-RestMethod : The underlying connection was closed: An unexpected error occurred on a send.

      + CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
      + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

      I have changed Expect100Continue to True.  However it appears that, there is some other type of 100 limit based on user/rate limit per min?  etc.  Please advise us, If there

       

      MaxIdleTime : 100000
      UseNagleAlgorithm : True
      ReceiveBufferSize : -1
      Expect100Continue : True
      IdleSince : 2/13/2020 3:35:26 PM
      ProtocolVersion : 1.1
      ConnectionName : https
      ConnectionLimit : 10
      CurrentConnections : 0
      Certificate : System.Security.Cryptography.X509Certificates.X509Certificate
      ClientCertificate :
      SupportsPipelining : True

      Thanks

      Hetal Soni

    • #203444
      Participant
      Topics: 12
      Replies: 1489
      Points: 1,987
      Helping Hand
      Rank: Community Hero

      Invoke-RestMethod : The underlying connection was closed: An unexpected error occurred on a send.

      Seen this error when the session ends and you are still passing it in the header. How does the API handle sessions? Is there a correlation of time rather than the number of records?

    • #203526
      Participant
      Topics: 1
      Replies: 2
      Points: 19
      Rank: Member

      Hi,

      Thank you very much for your response.  You are right at the point.  I have recreate script to break API call into batch of 90 and then add sleep for 90 seconds and then REAUTHNTICATE to API server and it seems to be working.

      However i would like to know, is there anyway from Wireshark or some sort of debug to see if Powershell really sending expect100continue in the header?  Does API server reject this request from client?   Is expect 100contiue is enabled, does that means , power shell doesn’t have to reauthenticate to api server?

      Thanks again for your support.

      Hetal soni

       

       

    • #203610
      Participant
      Topics: 18
      Replies: 575
      Points: 42
      Rank: Member

      Most api’s are rate limited. Try this.

      https://support.symantec.com/us/en/article.tech254111.html

       

    • #203661
      Participant
      Topics: 1
      Replies: 2
      Points: 19
      Rank: Member

      Thanks for the response. really appericiated.

      I have already raised value from 50 to 200  on SEP side, however as per SEP support, Powershell is restricting 100 API call, thus I have change Expect100Continue= True for Powershell-client side.  However I am  still seeing error @ 100 API call.

      For Expect100Continue — Do SEP Server have to support this ?  How do we verify that, Powershell ISE is sending Expect100Continue ?  through wireshar? or enabling some sort of debug on powershell side?

    • #203811
      Participant
      Topics: 12
      Replies: 1489
      Points: 1,987
      Helping Hand
      Rank: Community Hero

      All API’s do not work the same. They are going to have different rate limits, paging, record sizes, etc. that need to be taken into consideration. I’ve connected to hundreds of API’s and have never added a Expect100Continue in a header or anywhere else and collected a million records in single calls. There are common API vendors that software companies use, but the software company still needs to implement the features. It sounds like you are doing some kind of paging operation, how many records are you returning for each call? Is that adjustable? Explain the entire solution you are trying to engineer. There are X total records and your getting them at X records per call. The software vendor is going to be the best to assist you, they control the back-end and should have logging, but you can also tell them what you are doing and ask for guidance.

Viewing 5 reply threads
  • You must be logged in to reply to this topic.