2015-August Scripting Games Wrap-Up

The August puzzler was intended to highlight the usefulness of REST APIs, and the relative ease with which you can wrap PowerShell commands around them. Our Celebrity Entry for this month will be from Warren Frame, followed by the Official Entry from the puzzle's author, Don Jones.

Celebrity Entry

A short while back, I was plugging away on a fun PowerShell project, when I received a pleasant surprise - an invitation to write an article on the August Scripting Games Puzzle! For one reason or another, I’ve always missed out on the opportunity to participate in the Scripting Games, now I had no excuse.

I took notes as I read through the puzzle and began thinking and writing my solution; this article will be a stream-of-conscious on how I worked through this problem. Let’s start with the puzzle, paraphrased:

The Puzzle

Display data from www.telize.com/geoip in a table format, with longitude, latitude, continent_code, and timezone. The challenges were to solve this as a one-liner, as an advanced function, and to include notes on another handy web service you’ve used.

My first thought: Web APIs? Nice! I use these every day, and they’re quite important nowadays, with so many solutions offering a web API.

My second thought: Web APIs? Damn! This means I need to read through some documentation and figure out the intricacies of yet-another-REST-API.

Web APIs

Nowadays, everything but the kitchen sink comes with a web API, perhaps a RESTful API or a WSDL based web service. EMC Isilon, Citrix NetScaler, Microsoft Exchange, Thycotic Secret Server, Infoblox, StackExchange, and more. You name it, it probably has a web API.

This means learning how to call a web API from PowerShell can be incredibly valuable, allowing you to integrate and build tools and automation for these various technologies… But there’s a catch.

These APIs don’t have common conventions and standards to the extent that PowerShell does. They don’t have functions like Get-Help or Get-Member to tell you more about them; instead, they’re usually augmented with pages upon pages of documentation. Hopefully the vendor documented everything, and you find what you need.

Let’s look at some tools we can use for to solve this puzzle.

Web Services and PowerShell

We have a URL, and the puzzle mentioned JSON; there’s a good chance this is a REST API. Stephen Fox discussed some basic ways to differentiate these here. We have a number of tools at our disposal, including:

  • New-WebServiceProxy - Cmdlet used to connect to a traditional web service
  • Invoke-WebRequest - General purpose Cmdlet to send web requests and parse responses
  • Invoke-RESTMethod - Cmdlet to simplify calling REST web services
  • System.Net.WebClient - .NET class for more complex needs

The three Cmdlets were introduced in PowerShell 3.

Let’s step back a moment. The puzzle gave us a specific URL to query. This is very rare. In the wild, you’re going to run into the need to work with a particular technology. You’ll have to search around to see if there’s an API. There will be documentation. Maybe on the vendor’s website. Maybe with the installer bits. In this case, if we go to the root of http://www.telize.com/, we find the documentation and a few examples.

We have the ingredients for a solution: we know the PowerShell Cmdlets we can work with, and we have the documentation. Let’s fire up PowerShell and get to work!

Experimenting: The First Challenge

PowerShell is both a command line and scripting language. This means we can start experimenting with our ingredients right at the command line, even if our intent is to write a script or advanced function. Let’s try it out:

 

PS C:\> $Response = Invoke-WebRequest -Uri http://www.telize.com/geoip
PS C:\> $Response

Okay! That gave us back a bunch of stuff that we don’t need, but if we look at the Content property, we see some information we need for our output. Let’s expand this:

PS C:\> $Response.Content 

This looks like the data we need! It looks like a string though, who has time for parsing text? It turns out this is JSON, similar to XML, but much lighter weight. Starting in PowerShell 3, we have handy functions for converting JSON to objects and vice versa:

PS C:\> $Response.Content | ConvertFrom-Json 

That’s more like it! Thankfully, we have a shortcut that handles this under the hood: Invoke-RESTMethod.

PS C:\> Invoke-RESTMethod -Uri http://www.telize.com/geoip

Even simpler! Okay, let’s get nitpicky and format it for the challenge:

 

PS C:\> Invoke-RESTMethod -Uri http://www.telize.com/geoip |
Select-Object -Property longitude, latitude, continent_code, timezone 

First challenge down; let’s move on to something more useful: advanced functions.

Build Your Own Cmdlet: The Second Challenge

We won’t actually write and compile a Cmdlet, but we’ll write an advanced function, which looks and acts just like a Cmdlet. Our goal is to write Get-GeoInformation to provide the same output as above.

What’s an advanced function? Long story short, it’s just a function with the Cmdletbinding attribute. You can learn more about these in Learn PowerShell Toolmaking in a Month of Lunches, in this quick hit on best practices for PowerShell functions, or in about_Functions_Advanced.

No need to re-invent the wheel here, we can start with the built in Cmdlet (advanced function) snippet from the PowerShell ISE (version 3 or later). Alternatively, we can copy and paste from an advanced function we’ve already written.

We provide some help for the end user, and offer features like pipeline input and accepting arrays of IP addresses.

https://gist.github.com/RamblingCookieMonster/12547e3fbe29e78ce0dd

Two important notes to highlight from this code:

  • We want to take multiple IP addresses from the IPAddress parameter, so we need to loop over them. On the other hand, if no IPAddress is specified, we still need the code to run. We handle this by creating an IPAddress array in the Begin block, and loop over this if no IPAddress is specified.
  • We accept pipeline input like a normal Cmdlet. We do this by tagging the IPAddress parameter with ValueFromPipeline, and we have our code in the Process block, where the current pipeline item will be $IPAddress.

We can run Get-Help to see several ways to run our new command:

Get-Help Get-GeoInformation -Examples 

Let’s try it out.

Solving the second challenge:

PS C:\> Get-GeoInformation | Select-Object -Property longitude, latitude, continent_code, timezone 

 

Notice that we don’t restrict the output from the function. If we format or limit the output inside the function, we limit what we can do downstream. We could apply special formatting that doesn’t break this rule, but we omit this for simplicity.

That’s it! Creating re-usable tools as advanced function leaves you with more portable and readable code, and they’re fairly straightforward to write. Instead of copying and pasting snippets of code, consider writing these advanced functions, re-using them, and even sharing them with your co-workers and the PowerShell community

Could we do more?

The Third Challenge and Beyond

Back in the real world, you’ll find you need more than a single function for many web APIs, which might have a wealth of endpoints you can call. A PowerShell module is a great way to handle these.

For the third challenge, we’ll submit the StackExchange API. Notice how many endpoints there are! Many products in your environment may have similar functionality, allowing you to query and manage a system with web requests.

Perhaps you want to find unanswered PowerShell questions. Can you come up with code that provides the most recent 20 questions tagged PowerShell from stackoverflow, and format the output similar to this?

Stay tuned for a brief write-up on common practices around writing modules. We’ll illustrate this with the StackExchange API in an upcoming PowerShell.org article.

With so many web APIs out there, learning the basics can prove quite valuable. Take a look around your environment. Do you have any tedious manual tasks with products that have a web API? Try to build functions or modules for these!

Our time is up - two important closing notes:

  • Does your vendor provide a web API, but no PowerShell module? Remind them that an official PowerShell module would be very helpful.
  • Share your work with the community! If you start writing a module for product X and publish it on GitHub, you might find folks willing to contribute and improve your module. If we all write our own modules and keep them private, no one wins.

Cheers!

Official Entry

Whenever I'm given a REST API to work with, I often try to just hit the URL in my web browser. So I just ran over to www.telize.com/geoip. The result looked a lot like JSON - JavaScript Object Notation - and I'll freely admin that being able to distinguish between JSON and XML helped me a lot. Those are the two formats most web services will return.

Knowing that, I ran a quick Get-Help *JSON* in the shell and was rewarded with ConvertFrom-JSON.

Invoke-WebRequest http://www.telize.com/geoip | ConvertFrom-JSON

And that's essentially the answer. The JSON gets converted into PowerShell objects, and since there are only four properties it'll display in a table by default. Admittedly, to get the exact output from the puzzle, I had to add Format-Table -AutoSize to the end.

Turning this into an advanced function is equally straightforward, since you're really just wrapping some structure around the working answer.

function Get-GeoInformation {
    [CmdletBinding()]
    Param()
    Invoke-WebRequest http://www.telize.com/geoop | ConvertFrom-JSON
}

So, there it is. Yeah - I admitted comment-based help here, mainly to save space, but you hopefully get the idea! No Format-Table here, because functions shouldn't output pre-formatted text - doing so makes the data useless for anything else.

Your Reactions

First, please note that you need to post your code as a Gist, not just in a GitHub repo. The comment system will only retrieve actual Gists. Thanks.

Christian Sandaled came up with a much prettier advanced function, taking the time to include comment-based help, and allowing you to input any arbitrary IP address. That was definitely above and beyond, and it's a really well-written function, too. Jeff Bunting came up with something similar, also going so far as to allow for arbitrary IP address lookup. Good job, both of you! Others jumped in with similar solutions, adding error handling, and so on - definitely in the right spirit. Some folks - Adi, looking at you, friend - missed only slightly, by using a nonstandard function name. But still, great work.

The PowerShell Bear (grr) doubled down on difficulty by writing a PowerShell v2-compatible solution, which took Invoke-WebRequest off the table. Instead, the Bear used the underlying .NET Framework classes to do the job. He also had to manually unwind the JSON, as ConvertFrom-JSON was also unavailable in v2. Wow!

I didn't actually see any "bad" entries this month, which was extremely gratifying! It shows how much variation and individuality answers can have, while still adhering to good practices and form. And there's more than a few interesting new web services you'll discover in the comments, too!

 

Posted in:
About the Author

PowerShell.org Announcer

Profile photo of PowerShell.org Announcer

This is the official account for PowerShell.org and sponsor announcements.