Function with parallel runspaces shows strange behaviour

This topic contains 0 replies, has 1 voice, and was last updated by Profile photo of Forums Archives Forums Archives 5 years, 5 months ago.

  • Author
    Posts
  • #5996

    by megamorf at 2013-01-16 03:31:47

    Hey guys,

    I've written an advanced function over the past few days and I think it's useful in its current state. But I can't wrap my head around what's happening when the function is run multiple times in a row.

    Here's the function:

    Function Get-DateOfLastUpdate {

    [CmdletBinding()]
    param(
    [Parameter(Position = 0)]
    [String[]]
    $Computername,

    [Parameter(Position = 1)]
    [String]
    $Filepath,

    [Parameter(Position = 2)]
    [int]
    $ThrottleLimit = 10,

    [switch]$invoke
    )

    Write-Verbose "[$($myinvocation.mycommand)] :: Checking if AD module is loaded..."
    if(-not(Get-Module -Name ActiveDirectory))
    {
    try { Import-Module -Name ActiveDirectory }
    catch { return; Write-Error -message "Error loading ActiveDirectory module" }
    }

    if($computername)
    {
    Write-Verbose "[$($myinvocation.mycommand)] :: `Computername parameter is used..."
    Write-Verbose "[$($myinvocation.mycommand)] :: `$Computername contains [$($computername.count)] servers..."
    }
    else
    {
    Write-Verbose "[$($myinvocation.mycommand)] :: Retrieving servers from AD..."
    $computername = Get-ADComputer -filter "*" -searchbase "OU=Server,OU=Systemkonten,DC=ihkberlin,DC=intern" -properties "name","operatingsystem" |
    where{$_.operatingsystem -match "windows" } |
    Sort Name |
    Select -expandproperty name
    Write-Verbose "[$($myinvocation.mycommand)] :: Query returned [$($computername.count)] servers..."
    }

    Write-Verbose "[$($myinvocation.mycommand)] :: Preparing parallel processing of update timestamp..."
    $SessionState = [system.management.automation.runspaces.initialsessionstate]::CreateDefault()
    $Pool = [runspacefactory]::CreateRunspacePool(1, $throttleLimit, $SessionState, $Host)
    $Pool.Open()

    < # Return results from async runspaces http://msdn.microsoft.com/en-us/library ... 0;v=vs.85&#41;.aspx #>
    $ScriptBlock = {

    param($computer)

    $obj = "" | select computername, date
    $obj.computername = $computer

    if(test-connection $computer -count 1 -quiet)
    {
    $reg = [Microsoft.Win32.RegistryKey]::OpenRemoteBaseKey('LocalMachine', $computer)
    $date = [DateTime]( $reg.OpenSubKey("SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update\Results\Install") ).getvalue("LastSuccessTime")
    }
    else
    {
    $date = "---"
    }

    $obj.date = $date
    return $obj
    }

    $threads = @()
    $result = @()

    $handles = foreach($computer in $computername) {
    $powershell =
    ::Create().AddScript($ScriptBlock).AddArgument($Computer)
    $powershell.RunspacePool = $Pool
    $powershell.BeginInvoke()
    $threads += $powershell
    }

    do {
    $i = 0
    $done = $true
    foreach ($handle in $handles) {
    if ($handle -ne $null) {
    if ($handle.IsCompleted) {
    $result+= $threads[$i].EndInvoke($handle)
    $threads[$i].Dispose()
    $handles[$i] = $null
    } else {
    $done = $false
    }
    }
    $i++
    }
    if (-not $done) { Start-Sleep -Milliseconds 500 }
    } until ($done)

    if($Filepath)
    {
    $result | export-csv -notypeinformation -path $filepath -force
    if($invoke){ invoke-item $filepath }
    }

    return $result

    }

    When run a few times in a row it works fine but when I continue invoking the function my powershell suddenly tells me the following which roughly translates to 'Exception when calling EndInvoke with 1 argument. The thread was not started" and the rest of the lines say "The thread is running or was cancelled. Restart not possible". Ultimately, using normal cmdlets like clear-screen and get-childitem gives me an error in my powershell prompt after the function seemingly screwed up in the middle of closing the runspaces.
    I checked the scopes but everything the function does is contained in its local scope. The function shouldn't affect the parent scope, right?


    PS H:\> Get-DateOfLastUpdate -Verbose -Computername $comp
    VERBOSE: [Get-DateOfLastUpdate] :: Checking if AD module is loaded...
    VERBOSE: [Get-DateOfLastUpdate] :: Computername parameter is used...
    VERBOSE: [Get-DateOfLastUpdate] :: $Computername contains [25] servers...
    VERBOSE: [Get-DateOfLastUpdate] :: Preparing parallel processing of update timestamp...
    Ausnahme beim Aufrufen von "EndInvoke" mit 1 Argument(en): "Der Thread wurde nicht gestartet."
    Bei Zeile:70 Zeichen:45
    + $result+= $threads[$i].EndInvoke < <<< ($handle)
    + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : DotNetMethodException

    Ausnahme beim Aufrufen von "EndInvoke" mit 1 Argument(en): "Der Thread wird ausgeführt oder wurde abgebrochen. Neustart nicht möglich."
    Bei Zeile:70 Zeichen:45
    + $result+= $threads[$i].EndInvoke < <<< ($handle)
    + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : DotNetMethodException

    Ausnahme beim Aufrufen von "EndInvoke" mit 1 Argument(en): "Der Thread wird ausgeführt oder wurde abgebrochen. Neustart nicht möglich."
    Bei Zeile:70 Zeichen:45
    + $result+= $threads[$i].EndInvoke < <<< ($handle)
    + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : DotNetMethodException

    Ausnahme beim Aufrufen von "EndInvoke" mit 1 Argument(en): "Der Thread wird ausgeführt oder wurde abgebrochen. Neustart nicht möglich."
    Bei Zeile:70 Zeichen:45
    [... many more lines with the same error]

    computername : srvwts101
    date : 04.01.2013 22:04:24

    PS H:\> cls
    Der Thread wurde nicht gestartet.
    PS>cls
    Der Thread wurde nicht gestartet.
    PS>
    PS>
    PS>
    PS>gci
    Der Thread wurde nicht gestartet.
    PS>

    by nohandle at 2013-01-16 05:27:19

    It failing after multiple invocations of the command are run implies you hit some kind of limit. Check the task manager if there is enough memory etc.
    Also showing us the whole exception expanded ($error[index] | fl * -force) and resolved (all inner exceptions expanded), would probably help.

    btw:
    #does not work the cmdlet raises non-terminating error
    try { Import-Module -Name ActiveDirectory }
    catch { return; Write-Error -message "Error loading ActiveDirectory module" }

    #why the back tick before the C? you missed $?
    Write-Verbose "[$($myinvocation.mycommand)] :: `Computername parameter is used..."

    #why not including the operating system in the ldap filter?
    $computername = Get-ADComputer -filter "*" -searchbase "OU=Server,OU=Systemkonten,DC=ihkberlin,DC=intern" -properties "name","operatingsystem" |
    where{$_.operatingsystem -match "windows" }

    by megamorf at 2013-01-16 06:09:59

    [quote="nohandle"]It failing after multiple invocations of the command are run implies you hit some kind of limit. Check the task manager if there is enough memory etc.
    Also showing us the whole exception expanded ($error[index] | fl * -force) and resolved (all inner exceptions expanded), would probably help.
    [/quote]

    Okay, here are the full errors.

    This doesn't say much:

    PSMessageDetails :
    Exception : System.Management.Automation.MethodInvocationException: Ausnahme beim Aufrufen von "EndInvoke" mit 1 Argument(en): "Der Thread wird ausgeführt oder wurde
    abgebrochen. Neustart nicht möglich." ---> System.Threading.ThreadStateException: Der Thread wird ausgeführt oder wurde abgebrochen. Neustart nicht möglich
    .
    bei System.Management.Automation.Runspaces.AsyncResult.EndInvoke()
    bei System.Management.Automation.PowerShell.EndInvoke(IAsyncResult asyncResult)
    bei EndInvoke(Object , Object[] )
    bei System.Management.Automation.DotNetAdapter.AuxiliaryMethodInvoke(Object target, Object[] arguments, MethodInformation methodInformation, Object[] or
    iginalArguments)
    --- Ende der internen Ausnahmestapelüberwachung ---
    bei System.Management.Automation.DotNetAdapter.AuxiliaryMethodInvoke(Object target, Object[] arguments, MethodInformation methodInformation, Object[] or
    iginalArguments)
    bei System.Management.Automation.ParserOps.CallMethod(Token token, Object target, String methodName, Object[] paramArray, Boolean callStatic, Object val
    ueToSet)
    bei System.Management.Automation.MethodCallNode.InvokeMethod(Object target, Object[] arguments, Object value)
    bei System.Management.Automation.MethodCallNode.Execute(Array input, Pipe outputPipe, ExecutionContext context)
    bei System.Management.Automation.AssignmentStatementNode.Execute(Array input, Pipe outputPipe, ExecutionContext context)
    bei System.Management.Automation.StatementListNode.ExecuteStatement(ParseTreeNode statement, Array input, Pipe outputPipe, ArrayList& resultList, Execut
    ionContext context)
    TargetObject :
    CategoryInfo : NotSpecified: (:) [], MethodInvocationException
    FullyQualifiedErrorId : DotNetMethodException
    ErrorDetails :
    InvocationInfo : System.Management.Automation.InvocationInfo
    PipelineIterationInfo : {}

    Here's an interesting one

    PSMessageDetails :
    Exception : System.Management.Automation.MethodInvocationException: Ausnahme beim Aufrufen von "EndInvoke" mit 1 Argument(en): "Der Thread wurde nicht gestartet." ---
    > System.Threading.ThreadStartException: Der Thread wurde nicht gestartet. ---> System.OutOfMemoryException: Eine Ausnahme vom Typ "System.OutOfMemoryExcep
    tion" wurde ausgelöst.
    --- Ende der internen Ausnahmestapelüberwachung ---
    bei System.Management.Automation.Runspaces.AsyncResult.EndInvoke()
    bei System.Management.Automation.PowerShell.EndInvoke(IAsyncResult asyncResult)
    bei EndInvoke(Object , Object[] )
    bei System.Management.Automation.DotNetAdapter.AuxiliaryMethodInvoke(Object target, Object[] arguments, MethodInformation methodInformation, Object[] or
    iginalArguments)
    --- Ende der internen Ausnahmestapelüberwachung ---
    bei System.Management.Automation.DotNetAdapter.AuxiliaryMethodInvoke(Object target, Object[] arguments, MethodInformation methodInformation, Object[] or
    iginalArguments)
    bei System.Management.Automation.ParserOps.CallMethod(Token token, Object target, String methodName, Object[] paramArray, Boolean callStatic, Object val
    ueToSet)
    bei System.Management.Automation.MethodCallNode.InvokeMethod(Object target, Object[] arguments, Object value)
    bei System.Management.Automation.MethodCallNode.Execute(Array input, Pipe outputPipe, ExecutionContext context)
    bei System.Management.Automation.AssignmentStatementNode.Execute(Array input, Pipe outputPipe, ExecutionContext context)
    bei System.Management.Automation.StatementListNode.ExecuteStatement(ParseTreeNode statement, Array input, Pipe outputPipe, ArrayList& resultList, Execut
    ionContext context)
    TargetObject :
    CategoryInfo : NotSpecified: (:) [], MethodInvocationException
    FullyQualifiedErrorId : DotNetMethodException
    ErrorDetails :
    InvocationInfo : System.Management.Automation.InvocationInfo
    PipelineIterationInfo : {}

    Followed by Parsing errors that probably occur due to memory contraints

    ErrorRecord : Fehlende schließende "}" im Anweisungsblock.
    StackTrace : bei System.Management.Automation.Parser.ReportException(Object targetObject, Type exceptionType, Token errToken, String resourceIdAndErrorId, Obje
    ct[] args)
    bei System.Management.Automation.Tokenizer.Require(TokenId tokenId, String resourceIdAndErrorId, Object[] args)
    bei System.Management.Automation.Parser.ScriptBlockRule(String name, Boolean requireBrace, Boolean isFilter, ParameterDeclarationNode parameterDec
    laration, List`1 functionComments, List`1 parameterComments)
    bei System.Management.Automation.Parser.FunctionDeclarationRule()
    bei System.Management.Automation.Parser.StatementRule()
    bei System.Management.Automation.Parser.StatementListRule(Token start)
    bei System.Management.Automation.Parser.ScriptBlockRule(String name, Boolean requireBrace, Boolean isFilter, ParameterDeclarationNode parameterDec
    laration, List`1 functionComments, List`1 parameterComments)
    bei System.Management.Automation.Parser.ParseScriptBlock(String input, Boolean interactiveInput)
    bei System.Management.Automation.AutomationEngine.ParseScriptBlock(String script, Boolean interactiveCommand)
    bei System.Management.Automation.ScriptCommandProcessor..ctor(String script, ExecutionContext context, Boolean isFilter, Boolean useLocalScope, Bo
    olean interactiveCommand, CommandOrigin origin)
    bei System.Management.Automation.Runspaces.Command.CreateCommandProcessor(ExecutionContext executionContext, CommandFactory commandFactory, Boolea
    n addToHistory)
    bei System.Management.Automation.Runspaces.LocalPipeline.CreatePipelineProcessor()
    bei System.Management.Automation.Runspaces.LocalPipeline.InvokeHelper()
    bei System.Management.Automation.Runspaces.LocalPipeline.InvokeThreadProc()
    WasThrownFromThrowStatement : False
    Message : Fehlende schließende "}" im Anweisungsblock.
    Data : {}
    InnerException :
    TargetSite : System.Collections.ObjectModel.Collection`1[System.Management.Automation.PSObject] Invoke(System.Collections.IEnumerable)
    HelpLink :
    Source : System.Management.Automation

    [quote="nohandle"]
    btw:
    #does not work the cmdlet raises non-terminating error
    try { Import-Module -Name ActiveDirectory }
    catch { return; Write-Error -message "Error loading ActiveDirectory module" }

    #why the back tick before the C? you missed $?
    Write-Verbose "[$($myinvocation.mycommand)] :: `Computername parameter is used..."

    #why not including the operating system in the ldap filter?
    $computername = Get-ADComputer -filter "*" -searchbase "OU=Server,OU=Systemkonten,DC=ihkberlin,DC=intern" -properties "name","operatingsystem" |
    where{$_.operatingsystem -match "windows" }
    [/quote]

    Thanks for your input
    1) I'll rewrite that then
    2) The $-symbol is not there, I dunno why. thanks for spotting the typo 🙂
    3) I'm just getting into the AD module and LDAP filters. I'll include the OS in the LDAP filter instead

    PS: I tried changing the culture to en-US but i just wouldn't change (PSv2, WS 2008 R2 SP1)

    by megamorf at 2013-01-16 07:20:18

    I added a bit of logic to track the memory consumption but it doesn't seem like a lot to me – barely 3MB. Here's the modified part of the script (lines 80-96 of original function):

    $initial = (Get-Process -Id $PID).WorkingSet

    do {
    $memory = ((Get-Process -Id $PID).WorkingSet - $initial)
    [system.Console]::Title = ("Current Memory Load: {0:0.00} MB." -f ($memory/1MB))

    $i = 0
    $done = $true
    foreach ($handle in $handles) {
    if ($handle -ne $null) {
    if ($handle.IsCompleted) {
    $result+= $threads[$i].EndInvoke($handle)
    $threads[$i].Dispose()
    $handles[$i] = $null
    } else {
    $done = $false
    }
    }
    $i++
    }
    if (-not $done) { Start-Sleep -Milliseconds 500 }
    } until ($done)

    Also, my observations tell me that up until a throttlelimit of 5 the problem doesn't occur, so it really seems like a memory problem.

You must be logged in to reply to this topic.