Question on Pipelining and Splitting pipeline to different arrays

Welcome Forums General PowerShell Q&A Question on Pipelining and Splitting pipeline to different arrays

This topic contains 4 replies, has 3 voices, and was last updated by

 
Participant
3 months, 4 weeks ago.

  • Author
    Posts
  • #103237

    Participant
    Points: 0
    Rank: Member

    Hello Everyone,

    Powershell.org viewer for awhile, first time asker. I have a question and see if anyone to offer assitance on what I am trying to do or provide a better/different way of doing it.

    I am using VMWare.PowerCLI to get a list of VMs and some stats. I am keeping everything object and my last pipe line is a sort on one of the objects. So I have a large set of sorted objects. I want to pipe or foreach-object these objects in split them into four different arrays. Object.1 -> Array1[0] Object.2 -> Array2[0] Object.3 -> Array3[0] Object.4 -> Array4[0] Object.5 -> Array1[1] etc.

    The goal is I am trying to evenly distribute my collection of Objects into Four Smaller roughly equal Arrays or Objects.

    $vmusedspace = Get-VM | Select-Object Name, UsedSpaceGB | Sort-Object UsedSpaceGB -Descending
    
    $Job01 = @()
    $Job02 = @()
    $Job03 = @()
    $Job04 = @()
    

    $vmusedspace will have ~1000 objects in it sorted by UsedSpaceGB of the VM. I want $Job01 to contain 250 of the VMs, $job02 to contain 250 of the VMs, etc. With the UsedSpaceGB roughly the same between the four arrays.

    Thank you

  • #103243

    Participant
    Points: 0
    Rank: Member

    You can use Select-Object -First and -Skip to do paging.

    $job1 = $vms | Select -First 250
    $job2 = $vms | Select -Skip 250 -First 250
    
  • #103246

    Participant
    Points: 2
    Rank: Member

    Hmm. Well, first you need to know how many final objects you have.

    $SplitAmount = [int][math]::ceiling($VMUsedSpace / 4)

    Then you can divide it up into sets:

    $JobList = @{}
    $Index= 0
    do {
        $JobList[$Index] = $VMUsedSpace | Select-Object -First $SplitAmount
        $VMUsedSpace = $VMUsedSpace | Select-Object -Skip $SplitAmount # drop the ones you've just stored
        $Index++
    } while ($VMUsedSpace) # while there's still something in the original array
    foreach ($Job in $JobList.Values) {
        #Do stuff on each $Job
    }

    After that, you'll have a hashtable of arrays, effectively, each neatly portioned roughly in quarters. For any non-equal divisions, the last one will always be a tad short. Nothing serious.

    • #103253

      Participant
      Points: 0
      Rank: Member

      Thank you Joel Sallow and Rob Simmers. I did not know of the -First and -Skip options on Select-Object.

      However, neither of those will work for what I am looking for. Either of your code will put the first 250 objects in Aray1 and second 250 objects in Array2, etc.

      This would put all of my Large VMs in the first array because of my Sort on UsedSpaceGB. I am looking to evenly distribute or at least close the UsedSpaceGB across the Arrays.

      I need
      Object1 to go to Array1
      Object2 to go to Array2
      Object3 to go to Array3
      Object4 to go to Array4
      Object5 to go to Array1
      Object6 to go to Array2
      .
      .
      .

      Playing with Select-Object -Index I think I can get this to work. If someone could help me with the number manipulation part.

      $vmcount = (1..$vmusedspace.count) 
      

      Will get me an array of numbers with an index for every object. If I can get $job01Index = $vmcount MODULO 4 or something similar to work. Then my $job01Index should be {1,5,9,13,....) $job02Index = (2,6,10,14,.....) $Job03Index = (3,7,11,15,.....)

      $Job01 = $vmusedspace | Select-Object -Index $job01Index
      $job02 = $vmusedspace | Select-Object -Index $job02Index
      

      Any ideas on how to generate the IndexArrays? 1+4n, 2+4n, 3+4n, 4+4n.

      Thank you,

      Cory Fuchs

  • #103256

    Participant
    Points: 0
    Rank: Member

    Don't know if there is a better way, but the below code does work.

    $vmusedspace = Get-VM | Select-Object Name, UsedSpaceGB | Sort-Object UsedSpaceGB -Descending
    
    $totalvmcount = $vmusedspace | Measure-Object Name
    $vmcount =  $totalvmcount.Count
    $SplitAmount = [int][math]::ceiling($vmcount / 4)
    $job01index = (0..$SplitAmount) | ForEach-Object{0+4*$PSItem}
    $job02index = (0..$SplitAmount) | ForEach-Object{1+4*$PSItem}
    $job03index = (0..$SplitAmount) | ForEach-Object{2+4*$PSItem}
    $job04index = (0..$SplitAmount) | ForEach-Object{3+4*$PSItem}
    
    $Job01 = $vmusedspace | Select-Object -Index $job01index
    $Job02 = $vmusedspace | Select-Object -Index $job02index
    $Job03 = $vmusedspace | Select-Object -Index $job03index
    $Job04 = $vmusedspace | Select-Object -Index $job04index
    

    All of the VMs are in a job and evenly distributed between the jobs based off Used Space and still have objects.

    Thank you Joel and Rob for setting me down a path.

    If anyone has a more elegant solution please let me know.

    Cory Fuchs

The topic ‘Question on Pipelining and Splitting pipeline to different arrays’ is closed to new replies.