Pitfalls of the Pipeline
Pipelining is an important concept in PowerShell. Though the idea did not originate with PowerShell (you can find it used decades earlier in Unix, for example), PowerShell does provide the unique advantage of being able to pipeline not just text, but first-class .NET objects.
Pipelining has several advantages:
- It helps to conserve memory resources. Say you want to modify text in a huge file. Without a pipeline you might read the huge file into memory, modify the appropriate lines, and write the file back out to disk. If it is large enough you might not even have enough memory to read the whole thing.
- It can substantially improve actual performance. Commands in a pipeline are run concurrently-even if you have only a single processor, because when one process blocks, for example, while reading a large chunk of your file, then another process in the pipeline can do a unit of work in the meantime.
- It can have a significant effect on your end-user experience, enhancing the perceived performance dramatically. If your end-user executes a sequence of commands that takes 60 seconds, then until 60 seconds has elapsed he/she would see nothing without pipelining, whereas output could start appearing almost immediately with pipelining.
PowerShell provides a variety of techniques for using pipelining but it is all to easy to do it wrong, so you think you are pipelining but in fact you are not. In my article Ins and Outs of the PowerShell Pipeline, I discuss the most common things that can trip you up with implementing pipelining and how to avoid them.
Related Articles
Secure Your Powershell Session with JEA and Constrained Endpoints
Index What is a Constrained Endpoint and Why Would I Need One? Setup and Configuration Using our Endpoint What is a constrained endpoint and why would I need one? Powershell constrained endpoints are a means of interacting with powershell in a manner consistent with the principal of least privilege. In Powershell terms, this is referred to as Just-Enough-Administration, or JEA. JEA is very well documented, so this won’t simply be repeating everything those references detail.
Practical PowerShell Unit-Testing
By the time you are using PowerShell to automate an increasing amount of your system administration, database maintenance, or application-lifecycle work, you will likely come to the realization that PowerShell is indeed a first-class programming language and, as such, you need to treat it as such. That is, you need to do development in PowerShell just as you would with other languages, and in particular to increase robustness and decrease maintenance cost with unit tests and–dare I say–test-driven development (TDD).
Documenting your PowerShell API–solved!
Long has it been known how to easily document your PowerShell source code simply by embedding properly formatted documentation comments right along side your code, making maintenance relatively painless… But if you advanced to writing your PowerShell cmdlets in C#, you have largely been on your own, either hand-crafting MAML files or using targeted MAML editors far removed from your source code. But not anymore. With the advent of Chris Lambrou’s open-source XmlDoc2CmdletDoc, the world has been righted upon its axis once more: it allows instrumenting your C# source with doc-comments just like any other C# source: