Powershell endinvoke function gets stuck under heavy load

Tagged: 

This topic contains 1 reply, has 2 voices, and was last updated by Profile photo of Don Jones Don Jones 1 year, 8 months ago.

  • Author
    Posts
  • #23623

    My application is written in c# and uses Powershell class of .net library. It is multi-threaded and can execute a bunch of powershell scripts simultaneously on a runspacepool. I use a runspacepool size of 100. Before the execution of each script I create a session to the target computer, by using the code below. I use BeginInvoke to start execution in a pipeline and end-invoke to get the results, which include the session.


    ps = PowerShell.Create()
    var sessionScript = new PSCommand();
    sessionScript.AddCommand(New-PSSession);
    sessionScript.AddParameter("ConnectionUri", "http://192.xxx.x.xxx:5985";);
    sessionScript.AddParameter("AllowRedirection");
    var sessOpt = new System.Management.Automation.Remoting.PSSessionOption();
    sessOpt.OpenTimeout = new TimeSpan(0,0,0,0, 120000);
    sessOpt.CancelTimeout = new TimeSpan(0, 0, 0, 0,120000);
    sessionScript.AddParameter("SessionOption", sessOpt);
    var cred = new PSCredential(uName, password);
    sessionScript.AddParameter("Credential", cred);
    ps.Commands = sessionScript
    ps.RunSpacePool = mypool ## size 100 created earlier
    asyncResult = ps.BeginInvoke();
    resultsOut = ps.EndInvoke(asyncResult);

    All is fine when I keep my number of parallel executing sessions below 100. Once in a while when I hit 100 and above, I notice one or two threads (.net tasks) just getting stuck. I did some debugging and realized that the thread was getting stuck for EndInvoke. Even after a week the call just didn't return and the thread was still alive.

    So, I replaced execution code by adding a Timer to this operation

    Timer taskTimer = new Timer(TaskHitTimeout, null , 120000, System.Threading.Timeout.Infinite);
    asyncResult = ps.BeginInvoke();
    while (true)
    {
    if (asyncResult.IsCompleted || this.isCancelled)
    break;
    Thread.Sleep(200);
    }

    if (!this.isCancelled)
    resultsOut = ps.EndInvoke(asyncResult);

    and in the TimerHandler I set isCancelled to True


    private void TaskHitTimeout(Object statusCode)
    { this.isCancelled = True;}

    This did fix my problem. But I would like to understand why the call to EndInvoke just doesn't return. Is there anything else I can do to improve this. And If anyone else has seen a similar issue.

  • #23703
    Profile photo of Don Jones
    Don Jones
    Keymaster

    Most of us here are using PowerShell itself, rather than coding in in C#, unfortunately. Sorry that we haven't been able to help you to this point.

You must be logged in to reply to this topic.