Best way to support PoshRSJob inside of a function with pipelining

This topic contains 1 reply, has 2 voices, and was last updated by Profile photo of Dave Wyatt Dave Wyatt 4 months, 3 weeks ago.

Viewing 2 posts - 1 through 2 (of 2 total)
  • Author
    Posts
  • #38708
    Profile photo of Zuldan
    Zuldan
    Participant

    I'd like to know the best way I should be implementing runspaces (in my case, PoshRSJob) within a function that supports pipelining.

    I'm guessing Test-Pipeline1 is the most inefficient way of doing it. Is there a better way to do it that's not listed below?

    `

    function Test-Pipeline1
    {
        [CmdletBinding()]
        Param
        (
            [Parameter(ValueFromPipeline = $True)]
            $Data
        )
    
        Begin
        {
        }
        Process
        {
            $Data | Start-RSJob -ScriptBlock {
                if (1 -BAND $_){
                    "First ($_)"
                }Else{
                    Start-sleep -seconds (Get-Random -Minimum 1 -Maximum 10)
                    "Last ($_)"
                }
            } | Wait-RSJob -ShowProgress | Receive-RSJob | ForEach{"I am $($_)"}
        }
        End
        {
        }
    }
    
    function Test-Pipeline2
    {
        [CmdletBinding()]
        Param
        (
            [Parameter(ValueFromPipeline = $True)]
            $Data
        )
    
        Begin
        {
            $Batch = Get-Random
        }
        Process
        {
            $Data | Start-RSJob -Batch $Batch -ScriptBlock {
                if (1 -BAND $_){
                    "First ($_)"
                }Else{
                    Start-sleep -seconds (Get-Random -Minimum 1 -Maximum 10)
                    "Last ($_)"
                }
            } | Out-Null
        }
        End
        {
           Get-RSJob -Batch $Batch | Wait-RSJob -ShowProgress | Receive-RSJob | ForEach{"I am $($_)"}
        }
    }
    
    function Test-Pipeline3
    {
        [CmdletBinding()]
        Param
        (
            [Parameter(ValueFromPipeline = $True)]
            $Data
        )
    
        [string[]]$DataList = $null
        if ($Input.Count -eq 1) {
            # Get data list from $Data if pipeline not used
            $DataList = $Data
        } else {
            # Get data list from pipeline
            $DataList = $Input
        }
    
        $DataList | Start-RSJob -Batch $Batch -ScriptBlock {
            if (1 -BAND $_){
                "First ($_)"
            }Else{
                Start-sleep -seconds (Get-Random -Minimum 1 -Maximum 10)
                "Last ($_)"
            }
        } | Wait-RSJob -ShowProgress | Receive-RSJob | ForEach{"I am $($_)"}
    }
    
    1..10 | Test-Pipeline1
    1..10 | Test-Pipeline2
    1..10 | Test-Pipeline3
    

    `

    #38873
    Profile photo of Dave Wyatt
    Dave Wyatt
    Moderator

    I like option 2.

    In option 1, putting a Wait in the process block is bad when you're doing threading for performance, and option 3 doesn't have a process block at all.

    One thing that you might need to add at some point is a check for finished jobs in the Process block (but without Wait-RSJob). That way you can clean up the stuff in memory and send output objects down the pipeline without waiting for all of your pipeline input to finish. I haven't used this module yet, but I imagine it will look something like this:

    $completedJobs = Get-RSJob -Batch $Batch -State Completed
    $completedJobs | Receive-RSJob
    $completedJobs | Remove-RSJob
    
Viewing 2 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic.