Get-unique items from an array

Tagged: , ,

This topic contains 5 replies, has 3 voices, and was last updated by Profile photo of Stephen Yeadon Stephen Yeadon 2 years, 4 months ago.

  • Author
    Posts
  • #16851
    Profile photo of Stephen Yeadon
    Stephen Yeadon
    Participant

    Hi,

    I am trying to union two sets of data and get the distinct values back. The below code gets the top 5 processes by cpu and working set memory and unions the results together. I was expecting a distinct list of processes with the following id's ( 4, 900, 2536, 3196, 4452, 6924, 7404, 1524) as you can see the processes with the id's 7404 & 1524 are missing from the last result set that gets a unique set. I can not explain this. Any help would be appreciated.

    $process = Get-Process
    $cpu = $process | Sort-Object CPU -Descending | Select-Object -First 5
    $ws  = $process | Sort-Object WorkingSet -Descending | Select-Object -First 5
    
    $a = @()
    $a += $cpu
    $a += $ws
    
     
    $cpu
    
      NPM(K)    PM(K)      WS(K) VM(M)   CPU(s)     Id ProcessName
        629       0      108        308     3 2,802.36      4 System     
        171      15    15024      21636   106 2,334.49   4452 vmtoolsd   
       3306     321   615960     772640  1349 1,415.93    900 devenv     
        313      24    17740      25096   105 1,393.74   2536 vmtoolsd   
       3558     328   606116     766860  1368 1,356.66   6924 devenv     
    
     
    $ws
    
    Handles  NPM(K)    PM(K)      WS(K) VM(M)   CPU(s)     Id ProcessName
       2969     393   694852     883436  1581   652.15   3196 devenv     
       2621     341   628636     778132  1430   343.76   7404 devenv     
       3306     321   615960     772640  1349 1,415.93    900 devenv     
       3558     328   606116     766860  1368 1,356.66   6924 devenv     
       2732     344   590188     761144  1390   598.62   1524 de
    
    $a | Sort-Object id | Get-Unique
    
    Handles  NPM(K)    PM(K)      WS(K) VM(M)   CPU(s)     Id ProcessName
        629       0      108        308     3 2,802.36      4 System     
        171      15    15024      21636   106 2,334.49   4452 vmtoolsd   
       3306     321   615960     772640  1349 1,415.93    900 devenv     
        313      24    17740      25096   105 1,393.74   2536 vmtoolsd   
       3558     328   606116     766860  1368 1,356.66   6924 devenv     
    
  • #16852
    Profile photo of Simon Wåhlin
    Simon Wåhlin
    Participant

    Get-Unique compares each item in a list with the next item in the list and will only work against sorted lists.

    If you want to user Get-Unique you have to sort the list first:

    $a | Sort-Object -Property Id | Get-Unique
    

    Or you could use Select-Object to select unique entries from the list:

    $a | Select-Object -Unique
    
  • #16853
    Profile photo of Stephen Yeadon
    Stephen Yeadon
    Participant

    Hi Simon,

    I have tried both approaches and they both yield the same results. Also I am already sorting the array am I not

    $a | Sort-Object id | Get-Unique

    . Do you think that I need to sort the collection by something other than Id?

    Thanks

  • #16856
    Profile photo of Simon Wåhlin
    Simon Wåhlin
    Participant

    You are right, Select-Object -Unique and Get-Unique seems to both select unique entries based on ProcessName.

    To select unique entries based on only Id you could use this:

    $a | Sort-Object -Property Id -Unique
    

    If you want to select unique entries based on multiple values you would have to select all properties you want to use like this:

    $a | Select-Object 'Handles', 'NPM', 'PM', 'WS', 'VM', 'CPU', 'Id', 'ProcessName' -Unique
    
  • #16858
    Profile photo of Rob Simmers
    Rob Simmers
    Participant

    Try Sort-Object -Unique since it lets you indicate the property that you want to sort\be unique:

    $process = Get-Process
    $cpu = $process | Sort-Object CPU -Descending | Select-Object -First 5
    $ws  = $process | Sort-Object WorkingSet -Descending | Select-Object -First 5
     
    $a = @()
    $a += $cpu
    $a += $ws
    
    "Total object Count: {0}" -f $a.Count
    $a
    
    "Total unique ID's: {0}" -f ($a | Select ID -Unique).Count
    $a | Select ID -Unique | Format-Table -AutoSize
    
    $unique = $a | sort-object id -Unique
    "Total with Sort-Object -Unique:  {0}" -f $unique.Count 
    $unique

    Output:

    Total object Count: 10
    
    Handles  NPM(K)    PM(K)      WS(K) VM(M)   CPU(s)     Id ProcessName                                                       
    -------  ------    -----      ----- -----   ------     -- -----------                                                       
       3990     116   155344     183300   867 2,941.16   4124 OUTLOOK                                                           
       1529     135   152700     150592   594 2,420.08   3164 chrome                                                            
        221      59   180724     109000   358 2,208.63   2060 chrome                                                            
        564      15     4820       3520   107 1,258.55   2732 SynTPEnh                                                          
        428      32   143480     123344   460 1,190.34   3240 chrome                                                            
       1232     126   364880     313972  1147   955.64   6024 powershell_ise                                                    
       3990     116   155344     183300   867 2,941.16   4124 OUTLOOK                                                           
       3243     252   118944     162424   955   976.08   2692 explorer                                                          
       1529     135   152700     150592   594 2,420.08   3164 chrome                                                            
       2508      75   120028     130288   727   162.45   3368 lync                                                              
    Total unique ID's: 8
    
    
    
      Id
      --
    4124
    3164
    2060
    2732
    3240
    6024
    2692
    3368
    
    
    Total with Sort-Object -Unique:  8
    
    Handles NPM(K)  PM(K)  WS(K) VM(M)   CPU(s)   Id ProcessName   
    ------- ------  -----  ----- -----   ------   -- -----------   
        221     59 180724 109000   358 2,208.63 2060 chrome        
       3243    252 118944 162424   955   976.08 2692 explorer      
        564     15   4820   3520   107 1,258.55 2732 SynTPEnh      
       1529    135 152700 150592   594 2,420.08 3164 chrome        
        428     32 143480 123344   460 1,190.34 3240 chrome        
       2508     75 120028 130288   727   162.45 3368 lync          
       3990    116 155344 183300   867 2,941.16 4124 OUTLOOK       
       1232    126 364880 313972  1147   955.69 6024 powershell_ise

    Reference: http://blogs.technet.com/b/heyscriptingguy/archive/2012/01/15/use-powershell-to-choose-unique-objects-from-a-sorted-list.aspx

  • #16859
    Profile photo of Stephen Yeadon
    Stephen Yeadon
    Participant

    Thank you both Simon and Rob,

    Both solutions worked for me. Simon I cant believe that I didn't try:

    $a | Select-Object 'Handles', 'NPM', 'PM', 'WS', 'VM', 'CPU', 'Id', 'ProcessName' -Unique 

    I had previously tried :

    $a | Select-Object  -Unique 'Handles', 'NPM', 'PM', 'WS', 'VM', 'CPU', 'Id', 'ProcessName'

    which did not work for me.

    Rob I didn't even know that I could use -unique on a sort. That worked really well as using the select-object method did what I wanted but messed up the formatting which required me to format the table using expressions to tidy things back up.

    $a | Select-Object 'Handles', 'NPM', 'PM', 'WS', 'VM', 'CPU', 'Id', 'ProcessName' -Unique | 
            Format-Table @{n='Handles'; ex={$_.Handles}; align='right'; width=10} ,
                         @{n='NPM'; ex={$_.NPM}; align='right'; width=7}, 
                         @{n='PM'; ex={$_.PM}; align='right'; width=15}, 
                         @{n='WS'; ex={"{0:N2}" -f $_.WS}; align='right'; width=17},
                         @{n='VM'; ex={$_.VM}; align='right'; width=15}, 
                         @{n='CPU'; ex={"{0:N2}" -f $_.CPU}; align='right'; width=12},
                         @{n='Id'; ex={$_.Id}; align='right'; width=7}, 
                         @{n='ProcessName'; ex={$_.ProcessName}; align='right'; width=20} -wrap 
    

    vs

    $a | sort-object id -Unique
    

    Once again thanks both of you. I learned something new today.

You must be logged in to reply to this topic.