Author Posts

May 6, 2016 at 5:32 pm

I'd like to know the best way I should be implementing runspaces (in my case, PoshRSJob) within a function that supports pipelining.

I'm guessing Test-Pipeline1 is the most inefficient way of doing it. Is there a better way to do it that's not listed below?

`

function Test-Pipeline1
{
    [CmdletBinding()]
    Param
    (
        [Parameter(ValueFromPipeline = $True)]
        $Data
    )

    Begin
    {
    }
    Process
    {
        $Data | Start-RSJob -ScriptBlock {
            if (1 -BAND $_){
                "First ($_)"
            }Else{
                Start-sleep -seconds (Get-Random -Minimum 1 -Maximum 10)
                "Last ($_)"
            }
        } | Wait-RSJob -ShowProgress | Receive-RSJob | ForEach{"I am $($_)"}
    }
    End
    {
    }
}
function Test-Pipeline2
{
    [CmdletBinding()]
    Param
    (
        [Parameter(ValueFromPipeline = $True)]
        $Data
    )

    Begin
    {
        $Batch = Get-Random
    }
    Process
    {
        $Data | Start-RSJob -Batch $Batch -ScriptBlock {
            if (1 -BAND $_){
                "First ($_)"
            }Else{
                Start-sleep -seconds (Get-Random -Minimum 1 -Maximum 10)
                "Last ($_)"
            }
        } | Out-Null
    }
    End
    {
       Get-RSJob -Batch $Batch | Wait-RSJob -ShowProgress | Receive-RSJob | ForEach{"I am $($_)"}
    }
}
function Test-Pipeline3
{
    [CmdletBinding()]
    Param
    (
        [Parameter(ValueFromPipeline = $True)]
        $Data
    )

    [string[]]$DataList = $null
    if ($Input.Count -eq 1) {
        # Get data list from $Data if pipeline not used
        $DataList = $Data
    } else {
        # Get data list from pipeline
        $DataList = $Input
    }

    $DataList | Start-RSJob -Batch $Batch -ScriptBlock {
        if (1 -BAND $_){
            "First ($_)"
        }Else{
            Start-sleep -seconds (Get-Random -Minimum 1 -Maximum 10)
            "Last ($_)"
        }
    } | Wait-RSJob -ShowProgress | Receive-RSJob | ForEach{"I am $($_)"}
}
1..10 | Test-Pipeline1
1..10 | Test-Pipeline2
1..10 | Test-Pipeline3

`

May 10, 2016 at 7:34 am

I like option 2.

In option 1, putting a Wait in the process block is bad when you're doing threading for performance, and option 3 doesn't have a process block at all.

One thing that you might need to add at some point is a check for finished jobs in the Process block (but without Wait-RSJob). That way you can clean up the stuff in memory and send output objects down the pipeline without waiting for all of your pipeline input to finish. I haven't used this module yet, but I imagine it will look something like this:

$completedJobs = Get-RSJob -Batch $Batch -State Completed
$completedJobs | Receive-RSJob
$completedJobs | Remove-RSJob