Write-output going to console, not pipeline

This topic contains 0 replies, has 1 voice, and was last updated by  Forums Archives 5 years, 10 months ago.

  • Author
    Posts
  • #5536

    by Lembasts at 2013-02-11 18:46:10

    I have used write-output to send objects successfully to the pipeline.
    In each case I had one instance of an object.
    In a new cmdlet I am writing, I am processing a hash table from which I construct my object.
    However, I cannot write this resultant object to the pipeline as the cmdlet that will be reading the objects requires that they are sorted and I cannot guarantee that my keys in my original hash table are sorted.
    So I collect all my objects in an array using this line in my foreach processing :
    $newrows += $Dnsres | select *

    And then after the foreach loop I try this:
    $newrows = $newrows | sort computername
    $newrows | % {Write-Output $_}

    ...expecting each object to be pumped out to the pipeline in the correct order.
    Trouble is, it ends up displaying each object in the console and not delivering it to the pipeline.
    What is the bit I am missing here please?

    by DonJ at 2013-02-11 19:00:10

    Just run:

    $newrows

    Write-Output definitely writes to the pipeline. You're just mangling a lot of stuff there. Like, the default is to include all properties so I'm not sure what the Select * is for. Feels weird to be writing a command that just sorts. That's what Sort does :).

    by Lembasts at 2013-02-11 19:21:06

    Thanks Don.
    The line :
    $newrows += $Dnsres | select *
    ..is a technique I learned a while ago for collecting objects in an array. If you dont use the select *, then the resultant $newrows will only contain one instance of $dnsres, not all instances as processes through the loop. Its something to do with a pointer not being updated. Select * tricks it into updating some pointer. Im not sure about the internals – it just works.

    by Lembasts at 2013-02-11 20:02:06

    To show the above technique, run this first and you only get one set of values:
    $newobj = "" | select sum1,sum2,sum3
    $objcoll = @()
    $newobj.sum1 = 0
    $newobj.sum2 = 0
    $newobj.sum3 = 0

    for ($a = 1;$a -le 10;$a++) {
    $newobj.sum1 = $newobj.sum1 + $a
    $newobj.sum2 = $newobj.sum2 + $a * 2
    $newobj.sum3 = $newobj.sum3 + $a * 3
    $objcoll += $newobj
    }
    $objcoll

    ..but add the select * and it seems to work as expected:
    $newobj = "" | select sum1,sum2,sum3
    $objcoll = @()
    $newobj.sum1 = 0
    $newobj.sum2 = 0
    $newobj.sum3 = 0

    for ($a = 1;$a -le 10;$a++) {
    $newobj.sum1 = $newobj.sum1 + $a
    $newobj.sum2 = $newobj.sum2 + $a * 2
    $newobj.sum3 = $newobj.sum3 + $a * 3
    $objcoll += $newobj | select *
    }
    $objcoll

    by AlexBrassington at 2013-02-12 02:10:18

    If you just use the '+' operator then it all works.
    $objcoll + $newobj
    The reason it doesn't work with the += is interesting, the ' | select *' command creates a new object in memory, whose pointer is then added to the array so that's obvious enough. Without the ' | select *' you're just adding another copy of the same pointer to the array. What you probably expect is for it to identify this value has been used in multiple places and is now being modified, break the link, instantiate new objects on the heap and assign a new pointer to add to the array.

    I suspect that it might be because the $newObject is an object, itself a reference object type. In that case we've got pointers pointing to pointers, pointing to objects. In that case powershell isn't creating a new copy of the object becasue it's only referenced by one pointer. That pointer however is referenced by multiple pointers in the objColl array. The + operator seems to understand this and avoids the double pointer problem, possibly by creating two full chains to the object, however the += doesn't. Could powershell be asuming a pass by value rather than pass by reference?

    Of course i could be utterly wrong, references always do my head in, but i don't think i'm wrong about only needing the +. 🙂

    by mjolinor at 2013-02-12 04:32:12

    Actually, you don't need to do all that array addition at all:
    [code2=powershell]$newobj = "" | select sum1,sum2,sum3
    $objcoll = @()
    $newobj.sum1 = 0
    $newobj.sum2 = 0
    $newobj.sum3 = 0

    $objcoll =
    for ($a = 1;$a -le 10;$a++) {
    $newobj.sum1 = $newobj.sum1 + $a
    $newobj.sum2 = $newobj.sum2 + $a * 2
    $newobj.sum3 = $newobj.sum3 + $a * 3
    $newobj | select *
    }

    $objcoll[/code2]

    And looking at this a little more,
    [code2=powershell]$newobj | select *[/code2]
    works, but seems a rather "expensive" way to get a copy of the object.
    [code2=powershell]@($newobj).clone()[/code2]
    has the same effect, but appears to be a lot easieer on your processor (about 30x faster). Maybe something to consider inside a loop that's doing a lot of iterations.

    by Lembasts at 2013-02-12 13:15:58

    Thanks heaps for that. I do alot of this sort of coding where I have large chunks of code that set all the properties of an object, then collect those objects in an array.
    However my code is rather complex and I often have alot more than just a for loop to set properties. I call functions and stuff so assignment wont work ($objcoll = etc).
    So I tried:
    $objcoll + @($newobj).clone()
    and using measure-command it was three times faster than:
    $objcoll += $newobj | select *

    so thanks...

    by Lembasts at 2013-02-12 13:47:38

    One last observation. I found this statement on "stackoverflow":

    "Using += and + on arrays in PowerShell is making a copy of the array every time you use it. That is fine unless the list/array is really large. In that case, consider using a generic list:"

    As my arrays are huge I tried the following as another test:
    measure-command {
    $newobj = "" | select sum1,sum2,sum3
    #$objcoll = @()
    $objcoll = new-object System.Collections.Generic.List[Object]

    $newobj.sum1 = 0
    $newobj.sum2 = 0
    $newobj.sum3 = 0

    for ($a = 1;$a -le 10000;$a++) {
    $newobj.sum1 = $newobj.sum1 + $a
    $newobj.sum2 = $newobj.sum2 + $a * 2
    $newobj.sum3 = $newobj.sum3 + $a * 3
    $objcoll.add(@($newobj).clone())
    #$objcoll + @($newobj).clone()
    #$objcoll += $newobj | select *
    }
    }

    However it was slower than using "+" and the clone method.

    by mjolinor at 2013-02-12 15:18:56

    In my experience, adding to system.collections.arraylist is faster than adding to either arrays or generic collections.

    by Lembasts at 2013-02-12 17:44:40

    Never ending saga...

    Why does this work?
    $objcoll= new-object System.Collections.ArrayList
    $newobj = "" | select sum1,sum2,sum3
    $newobj.sum1 = 0
    $newobj.sum2 = 0
    $newobj.sum3 = 0

    for ($a = 1;$a -le 10;$a++) {
    $newobj.sum1 = $newobj.sum1 + $a
    $newobj.sum2 = $newobj.sum2 + $a * 2
    $newobj.sum3 = $newobj.sum3 + $a * 3
    $b = $newobj | select *
    $objcoll.add($b) | out-null
    }
    $objcoll

    but this doesnt?
    $objcoll= new-object System.Collections.ArrayList
    $newobj = "" | select sum1,sum2,sum3
    $newobj.sum1 = 0
    $newobj.sum2 = 0
    $newobj.sum3 = 0

    for ($a = 1;$a -le 10;$a++) {
    $newobj.sum1 = $newobj.sum1 + $a
    $newobj.sum2 = $newobj.sum2 + $a * 2
    $newobj.sum3 = $newobj.sum3 + $a * 3
    $b = @($newobj).clone()
    $objcoll.add($b) | out-null
    }
    $objcoll

    by mjolinor at 2013-02-12 18:30:03

    I swear I tested that and it worked. Now I can't get it to work either.
    After a little research I found this, which is not quite as good, but still much better than select | *:

    [code]$objcoll= new-object System.Collections.ArrayList
    $newobj = "" | select sum1,sum2,sum3
    $newobj.sum1 = 0
    $newobj.sum2 = 0
    $newobj.sum3 = 0

    for ($a = 1;$a -le 10;$a++) {
    $newobj.sum1 = $newobj.sum1 + $a
    $newobj.sum2 = $newobj.sum2 + $a * 2
    $newobj.sum3 = $newobj.sum3 + $a * 3
    $b = $newobj.psobject.copy()
    $objcoll.add($b) | out-null
    }
    $objcoll[/code]

    by Lembasts at 2013-02-12 18:39:59

    Thanks. We all learn this way 🙂

You must be logged in to reply to this topic.