Question on Pipelining and Splitting pipeline to different arrays

This topic contains 4 replies, has 3 voices, and was last updated by  Cory Fuchs 3 weeks, 3 days ago.

  • Author
  • #103237

    Cory Fuchs

    Hello Everyone, viewer for awhile, first time asker. I have a question and see if anyone to offer assitance on what I am trying to do or provide a better/different way of doing it.

    I am using VMWare.PowerCLI to get a list of VMs and some stats. I am keeping everything object and my last pipe line is a sort on one of the objects. So I have a large set of sorted objects. I want to pipe or foreach-object these objects in split them into four different arrays. Object.1 -> Array1[0] Object.2 -> Array2[0] Object.3 -> Array3[0] Object.4 -> Array4[0] Object.5 -> Array1[1] etc.

    The goal is I am trying to evenly distribute my collection of Objects into Four Smaller roughly equal Arrays or Objects.

    $vmusedspace = Get-VM | Select-Object Name, UsedSpaceGB | Sort-Object UsedSpaceGB -Descending
    $Job01 = @()
    $Job02 = @()
    $Job03 = @()
    $Job04 = @()

    $vmusedspace will have ~1000 objects in it sorted by UsedSpaceGB of the VM. I want $Job01 to contain 250 of the VMs, $job02 to contain 250 of the VMs, etc. With the UsedSpaceGB roughly the same between the four arrays.

    Thank you

  • #103243

    Rob Simmers

    You can use Select-Object -First and -Skip to do paging.

    $job1 = $vms | Select -First 250
    $job2 = $vms | Select -Skip 250 -First 250
  • #103246

    Joel Sallow

    Hmm. Well, first you need to know how many final objects you have.

    $SplitAmount = [int][math]::ceiling($VMUsedSpace / 4)

    Then you can divide it up into sets:

    $JobList = @{}
    $Index= 0
    do {
        $JobList[$Index] = $VMUsedSpace | Select-Object -First $SplitAmount
        $VMUsedSpace = $VMUsedSpace | Select-Object -Skip $SplitAmount # drop the ones you've just stored
    } while ($VMUsedSpace) # while there's still something in the original array
    foreach ($Job in $JobList.Values) {
        #Do stuff on each $Job

    After that, you'll have a hashtable of arrays, effectively, each neatly portioned roughly in quarters. For any non-equal divisions, the last one will always be a tad short. Nothing serious.

    • #103253

      Cory Fuchs

      Thank you Joel Sallow and Rob Simmers. I did not know of the -First and -Skip options on Select-Object.

      However, neither of those will work for what I am looking for. Either of your code will put the first 250 objects in Aray1 and second 250 objects in Array2, etc.

      This would put all of my Large VMs in the first array because of my Sort on UsedSpaceGB. I am looking to evenly distribute or at least close the UsedSpaceGB across the Arrays.

      I need
      Object1 to go to Array1
      Object2 to go to Array2
      Object3 to go to Array3
      Object4 to go to Array4
      Object5 to go to Array1
      Object6 to go to Array2

      Playing with Select-Object -Index I think I can get this to work. If someone could help me with the number manipulation part.

      $vmcount = (1..$vmusedspace.count) 

      Will get me an array of numbers with an index for every object. If I can get $job01Index = $vmcount MODULO 4 or something similar to work. Then my $job01Index should be {1,5,9,13,....) $job02Index = (2,6,10,14,.....) $Job03Index = (3,7,11,15,.....)

      $Job01 = $vmusedspace | Select-Object -Index $job01Index
      $job02 = $vmusedspace | Select-Object -Index $job02Index

      Any ideas on how to generate the IndexArrays? 1+4n, 2+4n, 3+4n, 4+4n.

      Thank you,

      Cory Fuchs

  • #103256

    Cory Fuchs

    Don't know if there is a better way, but the below code does work.

    $vmusedspace = Get-VM | Select-Object Name, UsedSpaceGB | Sort-Object UsedSpaceGB -Descending
    $totalvmcount = $vmusedspace | Measure-Object Name
    $vmcount =  $totalvmcount.Count
    $SplitAmount = [int][math]::ceiling($vmcount / 4)
    $job01index = (0..$SplitAmount) | ForEach-Object{0+4*$PSItem}
    $job02index = (0..$SplitAmount) | ForEach-Object{1+4*$PSItem}
    $job03index = (0..$SplitAmount) | ForEach-Object{2+4*$PSItem}
    $job04index = (0..$SplitAmount) | ForEach-Object{3+4*$PSItem}
    $Job01 = $vmusedspace | Select-Object -Index $job01index
    $Job02 = $vmusedspace | Select-Object -Index $job02index
    $Job03 = $vmusedspace | Select-Object -Index $job03index
    $Job04 = $vmusedspace | Select-Object -Index $job04index

    All of the VMs are in a job and evenly distributed between the jobs based off Used Space and still have objects.

    Thank you Joel and Rob for setting me down a path.

    If anyone has a more elegant solution please let me know.

    Cory Fuchs

You must be logged in to reply to this topic.