Multi-Threading Help

This topic contains 6 replies, has 3 voices, and was last updated by Profile photo of Warren Frame Warren Frame 1 year, 9 months ago.

  • Author
    Posts
  • #30590
    Profile photo of Joseph Monarch
    Joseph Monarch
    Participant

    I can't seem to get multi-threading to work properly. Can someone take a look and see what I am missing or doing incorrectly. The $code section actually works without multi-threading.

    $mobile = Get-ADComputer -SearchBase 'OU' -SearchScope 2 -Filter { (Name -notlike "MBV*") } | Select -ExpandProperty Name | Sort
    $path = "\directory"
    $outfile = "C:\Temp\ipaddress.txt"
    
    $RunspacePool = [runspacefactory]::CreateRunspacePool(1, 4)
    $RunspacePool.Open()
    
    $Code = {
    	param($pc)
    	$filenames = Get-ChildItem $path -Recurse | Where-Object { $_ -like "$pc*" } | Sort-Object -Property Name
    	if ($filenames -ne $null) {
    		$file = $filenames[-1]
    		
    		$ipstack = Get-Content "$path\$file" | Select-String -pattern "44\.", "172\." -List | group path | select group
    		$ipstack = $ipstack.Group
    		$ipstack = $ipstack | Out-String
    		$ipstart = $ipstack.IndexOf(";")
    		$ipstart = $ipstart + 1
    		$ipstring = $ipstack.Substring($ipstart)
    		$ipend = $ipstring.IndexOf(":")
    		$ip = $ipstring.Substring(0, $ipend)
    		"$ip`t$pc" | Out-File $outfile -Append
    	}
    }
    
    foreach ($pc in $mobile) {
    	$PSInstance = [powershell]::Create().AddScript($Code).AddArgument($pc)
    	$PSInstance.RunspacePool = $RunspacePool
    	$PSInstance.BeginInvoke()
    }
    

    Here is the output:

    >> Running (MultiThreading.ps1) Script...
    >> Platform: V4 64Bit (STA)

    CompletedSynchronously IsCompleted AsyncState AsyncWaitHandle
    ———————- ———– ———- —————
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEvent
    False False System.Threading.ManualResetEve

    >> Execution time: 00:00:02
    >> Script Ended

  • #30595
    Profile photo of Mark
    Mark
    Participant

    The first this I noticed is that you are attempting to use $path and $outfile in your $code section.

    If I recall correctly, Runspaces are separate from the main script and would have no knowledge of those variables.

    You'll have to pass those in.

  • #30598
    Profile photo of Joseph Monarch
    Joseph Monarch
    Participant

    Added this:

    param($pc, $path, $outfile)
    

    and this:

    $PSInstance = [powershell]::Create().AddScript($Code).AddArgument($pc).AddArgument($path).AddArgument($outfile)
    

    Same result as stated above.

  • #30600
    Profile photo of Mark
    Mark
    Participant

    I believe that the lines are a result of the BeginInvoke() command.

    You may want to add " | out-null" to the end of the line to suppress the output.

    One other potential problem is that each thread will attempt to append to the outfile on their own and they may overlap and error out.

    Your code doesn't store the BeginInvoke output, so it's not possible to perform "Pipe.EndInvoke" when the jobs complete to avoid the overlap.

  • #30601
    Profile photo of Joseph Monarch
    Joseph Monarch
    Participant

    The Out-Null did suppress the output. However, it is still not writing anything to the $outfile. Plus it is running in 00.00.01 seconds, so I don't think it is actually doing anything.

  • #30602
    Profile photo of Mark
    Mark
    Participant

    Actually, it is doing something.

    The ForEach loop is adding the jobs to the RunSpace queue. It doesn't mean that they're executing or have completed.

    I've generally following the method outlined on the following web site:

    I store the invocation information, wait until the jobs complete and scoop the output at the end.

  • #30610
    Profile photo of Warren Frame
    Warren Frame
    Participant

    Hi!

    I would recommend diving into Boe Prox' presentation (or his various previous bits) on using runspaces, if you're curious about working with these.

    That being said, there are a number of existing tools – Boe's PoshRSJob is the most polished, and gives you runspace performance using functions that behave like the *-Job Cmdlets. You could also try the Invoke-Parallel function, based on Boe's previous work.

    All this being said, whether you write your own or re-use an existing tool, please, *please* consider using abstraction through a function or module. Runspaces are incredibly useful, but leaving raw code like that in your script helps no one. You might understand it. I won't. Your co-workers might not. Substitute those many lines of code into a single, re-usable tool.

    Good luck either way!

You must be logged in to reply to this topic.