Tag Archives: Glenn Sizemore

PowerShell Great Debate: To Accelerate, or Not?


At his Birds of a feather session at TechEd 2013, Glenn Sizemore and I briefly debated something that I’d like to make the topic of today’s Great Debate. It has to do with how you create new, custom objects. For example, one approach – which I used to favor, but now think is too long-form:

$obj = New-Object -Type PSObject
$obj | Add-Member NoteProperty Foo $bar
$obj | Add-Member NoteProperty This $that

We saw some variants in The Scripting Games, including this one:

$obj = New-Object PSObject
Add-Member -InputObject $obj -Name Foo -MemberType NoteProperty -Value $bar

I generally don’t like any syntax that explicitly uses -InputObject like that; the parameter is designed to catch pipeline input, and using it explicitly strikes me as overly wordy, and doesn’t really leverage the shell.

Glenn and I both felt that, these days, a hashtable was the preferred approach:

$props = @{This=$that;
           Foo=$bar;
           These=$those}

The semicolons are optional when you type the construct that way, but I tend to use them out of habits that come from other languages. The point of our debate was that Glenn would use the hashtable like this:

$obj = [pscustomobject]$props

Because he feels it’s more concise, and because he puts a high value on quick readability. I personally prefer (and teach) a somewhat longer version:

$obj = New-Object -Type PSObject -Prop $props

Because, I argued, type accelerators like [pscustomobject] aren’t documented or discoverable. Someone running across your script can’t use the shell’s help system to figure out WTF is going on; with New-Object, on the other hand, they’ve got a help file and examples to rely on.

(BTW, I never worry about ordered hashtables; if I need the output in a specific order, I’ll use a custom view, a Format cmdlet, or Select-Object. A developer once explained to me that unordered hashtables are more memory-efficient for .NET, so I go with them).

But the big question on the table here is “to use type accelerators, or no?” You see this in many instances:

[null]Do-Something

# vs.

Do-Something | Out-Null

Same end effect of course, but I’ve always argued that the latter is more discoverable, while Glenn (and many others) prefer the brevity of the former.

So we’ll make today’s Great Debate two-pronged. What approach do you favor for creating custom objects? And, do you tend to prefer type accelerators, or no?

[boilerplate greatdebate]

Scripting Games Week 5


I loved this week’s challenge as it had the right wiggle room to bring out the best in our participants.  Of course, this is also the point in the games when we start to get everyone’s “A” game.  At this point even our new competitors are all warmed up and in the zone, and let me tell you the entries this week show it!   I want to start with the beginners as I actually ran almost every entry this week.  Honestly everyone fell into one of three buckets Select-string, Import-CSV or ,Foreach.  Let me explain there where three primary means to solve this problem.  Use Select-String and some basic text parsing to get the ip addresses, and then using Select-Object to filter.  Converting the logs to objects with Import-CSV and using Where-Object to filter.  Or using Foreach and a combination of if and where.

They are all three correct, so how does one judge one from another?  As this is a competition I used speed as the determining gauge.  For a long time I was convinced that the following was about perfect.  Quick simple and accurate.

Select-String -Path C:\Reporting\LogFiles\*\*.log -Pattern "(\b\d{1,3}\.){3}.\d{1,3}\b" -AllMatches | 
Select-Object -Unique @{Label="IP";Expression={$_.matches[1]}}

I was particularly drawn to this approach because it only used two cmdlets if that’s not PowerShell I don’t know what is. At first I was convinced converting the logs to objects was a waste.  Let me explain.  Over the course of this past month you’ve heard us rant and rave about objects, and how PowerShell is not text, but rich .Net objects.  For the most part that is an iron law, but it’s a law with an exception.  There is one place where text is just text, log files!  That’s why I loved this event.  This is the exception where all the old tricks still apply and where we found out which of you really know your regular expressions.  However in this one instant since we had a well formed log converting to a CSV was actually faster.   I wasn’t expecting that, but consider my gold standard example takes about 10 Seconds on my PC.   The Following finishes in 3!

$LogFilePath = 'C:\Reporting\LogFiles' 
$header = 'date','time','s-ip','cs-method','cs-uri-stem','cs-uri-query','s-port','cs-username','c-ip','cs(User-Agent)','sc-status','sc-substatus','sc-win32-status','time-taken' 
Import-Csv -Path $(Get-ChildItem -Path $LogFilePath -File -Recurse).FullName -Header $header -Delimiter ' ' | 
# if the contents of 'c-ip' can be converted to an IP address then it is a valid IP 
Select-Object @{n='ClientIP';e={if ([IPAddress]$_.'c-ip'){ $_.'c-ip' }}} | 
Sort-Object -Property 'ClientIP' -Unique

Now I’m not crazy about that entry it’s hard to follow, and will always return a blank string, but if you really look what makes it work is the author is offloading the IP filtering to the [IPAddress] type accelerator.  That is brilliant, and is x5 faster than a regular expression, which really adds up when you’re performing over 6k comparisons.   I know the general consensus is to leave the .Net stuff alone, but I have no religion when it comes to this stuff. If it’s better it’s better and in this instance it was better.

But that’s not the end of the story. While sorting through the entries I found the following solution.

Get-ChildItem -File C:\Reporting\LogFiles -Recurse | Get-Content | 
    # Selecting "GET /" gives us only the lines we want from the files.  
    Select-String -Pattern "GET /"  |
    # Split the remaining lines into an array and write element 8, the IP, to a file.
    ForEach-Object {$_.Line.Split("")[8] } | Select-Object -Unique @{Name="Source Address"; Expression={$_}}

Now that’s an old school PowerShell solution if I’ve ever seen one, and you know what it’s fast as hell!  There’s no validation of any kind. It will only work with provided source files, and it’s absolutely perfect!  You see the goal is to get the job done.  We don’t always have to author a tool that can be used by the world.  There is nothing wrong with leveraging your brain and cheating a little!

As for the advanced entries I think they’ve been adequately covered by my fellow judges.  In general my feedback would be to start a slow clap for the group.  There not perfect, but as a group you’ve learned from the feedback over this past month and man does it show! Heading into the final stretch I encourage you all to treat this last entry as your victory lap as you’ve all already one.

~Glenn

 

Scripting Games Week 4


Again if you’re participating in the games this year you’ve already won!  If you’re not and you’re reading this post what are you doing!  I’ve watched authors step there game up over the past month, and I can tell you from personal experience the games will make you better at your real job.  It’s like sharpening an axe, an axe made of super juice that can automate the world :)

Well that’s clever!
I came across this script this morning.

$prop = Write-Output Name,Title,Department,LastLogonDate,PasswordLastSet,LockedOut,Enabled
Get-ADUser -Filter * -Properties $prop | 
    Get-Random -Count 20 | Select-Object $prop |
        ConvertTo-Html -Title "Active Directory Audit" -PostContent "<hr>$(Get-Date)" | Out-File C:\adresult.html

Well formatted, simple concise, all around a very clean approach to the problem.  However the use of write-output threw me for a second.  I actually had to run it to see what was happening there, for a second I thought maybe there was yet another way to create a custom object in PowerShell.  Alas no, our intrepid author has simply deduced a way to avoid having to put quotes around the text.  Consider the following Prop1, and Prop2 are identical, but it’s one less character using write-output.

$prop1 = Write-Output Name,Title,Department,LastLogonDate,PasswordLastSet,LockedOut,Enabled
$prop2 = 'Name','Title','Department','LastLogonDate','PasswordLastSet','LockedOut','Enabled'

I’m not saying we should start using write-output instead of quotation if for nothing other than syntax highlighting it’s incorrect. However, this one time it’s forgiven, and I’m tipping my hat to you sir, well done.

Don’t put spaces or dashes in your property names.
I’ve seen this on and off throughout the games and I’ll admit this one isn’t a slam dunk, but that said don’t do it. You’re writing a script, camel case is the established standard for spaces. Yes the spaces do make it slightly easier to read, but at the cost of eliminate the reuse of the code.

Oh the Humanity.
Seriously read the damn help already. I could just fill this post with examples of simple mistakes that could have been avoided. Using the wrong cmdlet is one thing but take the following.

Get-Process | Sort-Object {Get-Random} | select -First 5

What’s wrong with that picture? Well nothing except it’s horribly inefficient since the Get-Random cmdlet has a count parameter!

Get-Process | Get-Random -Count 5

To the author You know who you are, everyone else read the help people!

Light week this week, but I will say I am super excited about next weeks offerings it’s a problem that tickles my kind of fancy, and I hope you all have as much fun solving it as I did.

~Glenn

Scripting Games Week 2: Formatting edition


This time of the year always feels like someone is holding down the fast forward button.  I blinked and here we are Friday morning another week of scripts in the rear view.  I spent most of my week in the beginner class this week, and was greeted by a combination of beginners and scripters who weren’t quite ready to step up to advanced.  More of the latter if I’m to be honest.  This was a pleasant surprise as it’s another sign of the continuing growth of our community.  Now on to the scripts I knew when I signed up to do this, that at least one of these weeks I’d talk about formatting.  It’s one of those best practices that you don’t appreciate until you’re asked to review someone else’s code.

Don’t Crunch the Code, and for the love of all things, Hit Enter!
I did not deduct any points for readability, but you didn’t make my good list either.  Personally I find it disrespectful to share an ungodly one-liner, but it’s downright wrong if that single line has semicolons!  We’re not printing these scripts the crunch gets us nothing. I’m not going to call out the litany of scripts that were manually formatting the data directly which is even worse, but consider the following.

Get-Content C:\IpList.txt | Foreach-Object { $Processor = Get-WmiObject -ComputerName $_ -NameSpace "Root\CIMV2" -Class "Win32_Processor"; $OpSystem  = Get-WmiObject -ComputerName $_ -Namespace "Root\CIMV2" -Class "Win32_OperatingSystem"; New-Object -TypeName PSObject -Property @{ Name = $Processor.SystemName; Cores = $Processor.NumberOfCores; OS = $OpSystem.Caption; Version = $OpSystem.Version; Memory = $OpSystem.TotalVisibleMemorySize }  }

This is an almost perfect solution and it’s utilizing my next tip for this week already, but the formatting made it unnecessarily hard to read. Let just clean this up a bit by inserting a proper CR in place of all those semi-colons.

Get-Content C:\IpList.txt | Foreach-Object { 
    $Processor = Get-WmiObject -ComputerName $_ -Class "Win32_Processor"
    $OpSystem  = Get-WmiObject -ComputerName $_ -Class "Win32_OperatingSystem"
    New-Object -TypeName PSObject -Property @{ 
        "Name"    = $Processor.SystemName
        "Cores"   = $Processor.NumberOfCores
        "OS"      = $OpSystem.Caption
        "Version" = $OpSystem.Version
        "Memory"  = $OpSystem.TotalVisibleMemorySize 
    }  
}

Does anyone honestly not think the latter is better? The whitespace cost nothing at execution, and makes it an order of magnitude easier for a human being to read, process, and comprehend! I don’t care what you do in your own scripts but when another human being is going to be asked to read it take a moment and format it. By the way for those in audience in love with the all-powerful one-liner both those examples are one-liners.
That’s Sooooo 2006!
Seriously, it’s okay to use the latest features of the language! Heck how about we just agree to use the features from the last version! What am I talking about? object creation! Again I didn’t take any points off for this, and you may have made my good list, but I didn’t like it. Select-Object and Add-Member NoteProperty were how we built custom object in 2006 with PowerShell v1. PowerShell V2 added an extremely powerful –Property parameter to New-Object that completely removed the need for Add-Member, and PowerShell V3 introduced the [PSCustomObject] type accelerator that removed them all! Consider the following look back at the past six years of PowerShell Object Creation.

# 2006
Get-Content .\IpList.txt | Foreach-Object { 
    $Processor = Get-WmiObject -ComputerName $_ -Class "Win32_Processor"
    $OpSystem  = Get-WmiObject -ComputerName $_ -Class "Win32_OperatingSystem"
    New-Object -TypeName PSObject |
        Add-Member -MemberType Noteproperty -Name "Name" -value $Processor.SystemName -PassThru | 
        Add-Member -MemberType Noteproperty -Name "Cores" -value $Processor.NumberOfCores -PassThru | 
        Add-Member -MemberType Noteproperty -Name "OS" -value $OpSystem.Caption -PassThru | 
        Add-Member -MemberType Noteproperty -Name "Version" -value $OpSystem.Version -PassThru |
        Add-Member -MemberType Noteproperty -Name "Memory" -value $OpSystem.TotalVisibleMemorySize -PassThru
} 

# 2007 This worked in 2006, but it took a little while to catch on.
Get-Content .\IpList.txt | Foreach-Object { 
    $OpSystem  = Get-WmiObject -ComputerName $_ -Class "Win32_OperatingSystem"
    Get-WmiObject -ComputerName $_ -Class "Win32_Processor"|  
        Select-Object -Property SystemName, NumberOfCores,
            @{'Name'="OS";"Expression"={$OpSystem.Caption}},
            @{'Name'="Version";"Expression"={$OpSystem.Version}},
            @{'Name'="Memory";"Expression"={$OpSystem.TotalVisibleMemorySize}}
} 

# 2009
Get-Content .\IpList.txt | Foreach-Object { 
    $Processor = Get-WmiObject -ComputerName $_ -Class "Win32_Processor"
    $OpSystem  = Get-WmiObject -ComputerName $_ -Class "Win32_OperatingSystem"
    New-Object -TypeName PSObject -Property @{
        "Name"   = $Processor.SystemName
        "Cores"  = $Processor.NumberOfCores
        "OS"     = $OpSystem.Caption
        "Version"= $OpSystem.Version
        "Memory" = $OpSystem.TotalVisibleMemorySize
    }
} 

# 2012
Get-Content .\IpList.txt | Foreach-Object { 
    $Processor = Get-WmiObject -ComputerName $_ -Class "Win32_Processor"
    $OpSystem  = Get-WmiObject -ComputerName $_ -Class "Win32_OperatingSystem"
    [PSCustomObject]@{
        "Name"   = $Processor.SystemName
        "Cores"  = $Processor.NumberOfCores
        "OS"     = $OpSystem.Caption
        "Version"= $OpSystem.Version
        "Memory" = $OpSystem.TotalVisibleMemorySize
    }
}

They are all more or less the same. When properly formatted they are all equally readable. Most of them use a hash table of some sort. Therefor there are some language hurdles that need to be cleared, so why bother, why does it matter?… simple performance, with every release the PowerShell team have refined Object creation and the new way is always just a little bit faster. I used measure-command to measure the execution times for the above examples and well as you can see while minute every subsequent technique is slightly faster.

New-Object/Add-Member = 1128 Milliseconds
Select-Object                    = 1114 Milliseconds
New-Object –property      = 1107 Milliseconds
PSCustomObject             = 1100 Milliseconds

Again not a huge deal but given a large enough dataset every tick counts. There were a litany of other things that I saw this week that made my list. The good news is this is all nitpicky stuff which is awesome!   Keep it up, and for the rest of you voters out there lets ease up with the ones and twos these are awesome scripts.  They may not use the technique you’d prefer but for the most part they’re getting the job done.

~Glenn

OK i’m impressed: Scripting Games Week 1


Well guys, and gals another year has passed, and the annual scripting games are upon us again.  After a week of reviewing submissions for their technique and style I must say that I am truly impressed!  As a community the average ability seems to be growing by leaps and bounds.  That’s not to say we’re all Samurai just yet, but we’re getting there!

Before I go off and nit-pick I want to congratulate you all on a small mountain of really well written scripts.  Some of the things that the community was preaching 5 years ago are now just standard.  Stuff like comment your code, format for readability, and Parameters.  At this point I’m convinced those who still aren’t conforming are simply non-conformist and well that’s a lost cause.  For the rest of us great work and keep it up!

Where is the Help!
What I didn’t see enough of in the advanced category is help.  Honestly if you’re going to write a 200 line script fill out the help!  It’s not that hard and it is THE difference between a good script and a great solution! It’s also one of the fundamental differences between hacking and tool building, both are focused around automating a given problem set.  The hacker just gets it to work, the tool builder makes it usable by the masses.  If you haven’t figured it out yet the real money is in tool building, I’m just sayin!

Trust but Validate.
I was pleasantly surprised by the amount of error handling in this first round of submissions, however I was disappointed by the lack of parameter validation.  When done correctly parameter validation can remove most of the potential errors a script can run into, and the best part is you find out that it’s not going to work before the script does anything!  For example in this week’s scenario every single script was asked to supply a source and destination path.  The following would have removed all but an access denied error.

Param ( 
    [Parameter(Mandatory=$true, ValuefrompipelineByPropertyName=$true)]
    [ValidateScript({Test-Path $_ -PathType Container})] 
    [Alias("FullName")]
    [string]$Source
,
    [Parameter(Mandatory=$true, ValuefrompipelineByPropertyName=$true)]
    [ValidateScript({Test-Path $_ -PathType Container})] 
    [Alias("FullName")]
    [string]$Destination
)

This is the equivalent of filter to the left, and I’ve talked to endless developers who are a little jealous of our ability to use an arbitrary scriptblock for parameter validation. For more static values the ValidateSet attribute will perform the same function, but with the added benefit of Intelli-sense and tab completion. Guys use this I’m telling you it’s one of the most powerful features in PowerShell and I just don’t see it use often enough, but then again I’ve been tilting at this windmill for years now.

Parameter names
This one is a little more nitpicky than the average, but honestly there simply isn’t an excuse for a script with three parameters to all start with the same letter.  Meaning the following is just disrespectful to yourself and your users.

Param(            
    [String]$ArchiveSource,            
    [String]$ArchiveDestination,            
    [String]$ArchiveAge            
)

I mean that’s a no-brainer right?  I don’t assume malice here just a lack of focus.  Anyone who stops and thinks about it immediately sees the problem, and solution. So I guess what I’m asking is that we collectively take a second to think about usability.  For those of you that haven’t had your coffee yet. The solution is that since three parameters all contain Archive we need to move that bit from the beginning of each parameter name.   In this case since there is no real need to differentiate I would suggest removing it all together.

Param(            
    [String]$Source,            
    [String]$Destination,            
    [String]$Age            
)

Here we’re focusing on what’s really important which makes the parameters easier to comprehend, but also lets us get to TAB faster which is a huge part of usability!

Bring it in
In summary all in all I would say we had a fantastic showing for our industry this initial week.  I really like the new site and voting has been very productive which is nice.  As we head into week two I look forward to what’s to come as we collectively build upon what we’ve learned this week.

~Glenn

Up Next: Glenn Sizemore from NetApp


On this week’s show (6/15/11 @ 9:30pm EST right here on Justin.tv!), we’ll be talking to Glenn Sizemore from NetApp about his latest PowerShell stuff announced and demo’d recently at TechEd. Be sure to drop by the chatroom Wednesday so that you can post questions to Glenn and interact with your fellow PowerShellers!