PowerShell.org's Azure Journey, Part 1

When we started PowerShell.org, my company (Concentrated Tech) donated shared hosting space to get the site up and running. We knew it wouldn't be a permanent solution, but it let us start out for free. We're coming to the point where a move to dedicated hosting will be desirable, and we're looking at the options. Azure and Amazon Web Services are priced roughly the same for what we need, so as a Microsoft-centric community Azure's obviously the way to go.

Azure Technical Fellow Mark Russinovich is having someone on his team connect with me to discuss some of the models in which we could use Azure. What makes the discussion interesting is that PowerShell.org runs on a LAMP (Linux, Apache, MySQL, and PHP) stack. We're not looking to change that; WordPress requires PHP, and the Windows builds of PHP typically lack some of the key PHP extensions we use. I'm not interested in compiling my own PHP build, either - I want off-the-shelf. WordPress more or less requires MySQL; while there's a SQL Server adapter available, it can't handle plugins that don't use WordPress' database abstraction layer, and I just don't want to take the chance of needing such a plugin at some point and not being able to use it.

What's neat about Azure is that it doesn't care. I adore Microsoft for selling a service and not caring what I do with it. Azure runs Linux just fine. Huzzah!

So, we've got two basic models that could work for us. Model 1 is to just buy virtual machines in Azure. We're planning one for the database and another for the Web site itself, so that we can scale-out the Web end if we want to in the future. We're not going to do an availability set; that means we risk some short downtime if Azure experiences hardware problems and needs to move our VM, but we're fine with that because right now we can't afford better availability. We'd probably build CentOS machines using Azure's provided base image (again, adore Microsoft for making this easy for Linux hosting and not just Windows). We know we tend to top out at 250GB of bandwidth a month, and that we need about 1GB of disk space for the Web site. 500MB of space for the database will last us a long time, but we'd probably get 1GB for that, too. It's only like $3 a month. We could probably start with Small VM instances and upgrade later if needed. All-in, we're probably looking at about $125/mo, less any prepay discounts.

Model 2 is to just run a Website. We still get to pick the kind of instance that hosts our site, so if we went with Small and a single instance, we'd be at about $110 including bandwidth and storage. That doesn't include MySQL, though. Interestingly, Microsoft doesn't host MySQL themselves as they do with SQL Azure. Instead, they outsource to ClearDB.com, which provides an Azure-like service for hosted MySQL. Unfortunately, the Azure price calculator doesn't cover the resold ClearDB service. Looking at ClearDB's own pricing, it'd probably push us to about $120-$125 a month - or about the same as having our own virtual machines. The difference is that, with Model 2, Microsoft can float our Web site to whatever virtual hosts they need to at the time to balance performance; with Model 1, they can potentially move our entire VM - although they're unlikely to do so routinely, since it'd involve taking us offline for a brief period. A super-neat part of this model is its integration with Git: I can run a local test version of the site, and as I make changes and commit them to our GitHub repository, Azure can execute a pull and get the latest version of the site code right from Git. Awesome and automated. I love automated.

An appeal of Model 1 is that I can build out the proposed CentOS environment on my own Hyper-V server, hit it with some test traffic loads, and size the machine appropriately. I can then deploy the VHDs right to Azure, knowing that the instance size I picked will be suitable for the traffic we need to handle. It also give me an opportunity to validate the fact that a dedicated VM will be faster than our current shared hosting system, and to play around with the more advanced caching and optimization options available on a dedicated VM. I can get everything dialed in perfectly, and then deploy.

Azure has other usage models, but these are the two applicable to us. I think it's great that we get these options, and that the pricing is more or less the same regardless. And again, I think it's pure genius that Azure's in the business of making money for Microsoft, and that they're happy to do so running whatever OS I want them to.

I'll continue this series of posts as we move through the process, just for the benefit of anyone who's interested in seeing Azure-ification from start to finish. Let me know if you have any questions or feedback!

About the Author

Don Jones

Profile photo of Don Jones

Don Jones is a Windows PowerShell MVP, author of several Windows PowerShell books (and other IT books), Co-founder and President/CEO of PowerShell.org, PowerShell columnist for Microsoft TechNet Magazine, PowerShell educator, and designer/author of several Windows PowerShell courses (including Microsoft's). Power to the shell!

17 Comments

  1. PowerShell on Linux exists to help Windows admins to help them transition away from Windows Server. But In a decade, when > 75% of workloads running on Azure are Linux, the tools used to manage those workloads will be designed for and meet the needs of the Linux administrator mindset. Linux administrators have a great set of mature tools and it isn't entirely clear why they'd want to transition to using PowerShell. Linux has won. The growth of Linux on Azure shows that. It's time for Azure to be a Linux First environment - and PowerShell on Linux doesn't really fit because it's primarily trying to graft a new and unproven Microsoft way of doing things onto successful practices and culture. Sure they can do it - but they'd be better off retraining everyone to "Think Linux".

    If Microsoft has (apparently) given up on Windows Server and understand that only a few legacy holdouts will want to run it in Azure in 10 years time (with the majority of workloads being Linux), they are wasting resources trying to reinvent how Linux administrators do things as a way of placating legacy Windows administrators. Legacy Windows administrators should bite the bullet and go "all-in" and adopt existing successful open source administration paradigms. The clock is ticking on their relevance and if they spend precious time investing in a nascent administration technology instead of fully transitioning to an open source mindset, they'll be less employable in future.

    PowerShell on Linux would be a neat idea if Windows Server had a future. It'll remain around in the same way that mainframes are still with us - but Microsoft has no interest in making a compelling case for organizations to choose their product over the free alternative. The future of Windows Server is the current reality of Windows Phone.

    • Thank you SO much for contributing your perspective! I do think - and I'm not a Microsoft fanboy per se - that you're misinformed, or at least under-informed. Time will tell, of course, but I suspect you've brought some personal bias to your viewpoint.

    • Interesting perspective. But I think you missed the point why Microsoft is actually open sourcing PowerShell. If you followed the talks that Jeffrey Snover did the last few years, it became clear that they want to be able to support heterogeneous environments. And for what I've seen now from Microsoft, and especially the PowerShell team, is that they don't have a hidden agenda. Microsoft is not the Microsoft anymore from let's say, 5-10 years ago.

      I agree with you, that there's a lot of Linux on Azure. But many, many companies I come are mainly Windows based infrastructures. Also guys that I know that work for other companies almost only see Windows based infra's.

      Yes, Linux has it's place in this world, but according for Microsoft there's no battle between Windows or Linux. They're citizens in IT which Microsoft wants to support best. And PowerShell is not a tool per se, it's meant to be a management framework. A framework that operates with built-in tools or in the case of Windows, the .NET framework.

      Windows Server has a big future, just look at the developments Microsoft is doing on Nano server.

    • Do you really think in 20 years well be dealing with "OS war" ?

      Do you think well see linux or windows in 20-30 years ? I dont.

      Do you think there is a loosing side or a winning side ? There are never winners in any war.

      From my POV, the shift towards lean kernels to accommodate the cloud, will get us eventually to a unified kernel of some sort getting the best of breed of all OSes, giving developers and IT the option to focus on the tools and the frameworks and less about the underlying layers.

      Running a business that creates OS is becoming very expensive. No one wants to be limited in the tools they want to use, thus the SQL on Linux is a huge huge thing in that sense and it will only get bigger with more such products going the same way. I have yet to see any major party offer anything similar things, coming from the Linux side because it takes money and effort very little companies have, so MS in that sense is helping transform the ecosystem again and its in a very good direction.

      Powershell on Linux exists so I, as a windows admin will have a lower barrier of entrance, if my boss decides one day to invest some our company assets on linux. If I can help my company get the right decisions that will save it money and achieve more and if that means going with a non MS way, guess what, I can still use my skills from the windows side without the hassle of the learning curve.

      For a long time I've been an advocate of learning both windows and Linux, no matter what I do mostly in my work time, as they are just tools to make the job, means to achieve a goal..they are not the goals themselves, and the movement to the cloud just emphasize it even more.

      I think your notion of what open source and free means is what's leading you in the line of thought and that's where I think you were wrong, imho.
      Not saying that my notion of what open source and free means is better, but its somewhat less biased. Nothing is free. Open source doesn't mean security (look at the horrible OpenSSL hole that's been there for two years and only recently been closed, or support-when-you-NEED it, that will always cost money, either by support contracts or having devs that know that specific language to deal with the bugs internaly (which by itself is even more limiting with the amount of languages and frameworks popping every second day).

      As for hidden agendas, you need to remember this is still a business. There's always money involved and business opportunities to be made. MS along they years was always good in creating those opportunities for itself and its partners and it continues to do so, the bottom line will be the tools. If you have ones that do the job for you, keep using them. If MS puts money and effort to create better tools with the community, who's the winner ? Everyone.

      I've seen this in the heated arguments on the PS repo the second it went public. The lack of broader vision some of the Linux base audience showed, the arrogance, the "Its mine, dont touch it" is somewhat alarming. I'm just happy to know that the sysadmins in 20 years, the ones born today well have a different starting point where they will choose the tools and be told what to use by old retiring sysadmins that are trying to hold to their precious seats instead of embracing change and supporting it in the evolving it world.

  2. If I have to copy files from installation folder to destination and I have to do exception handling because everything is automated, then how to do that? What all errors may arise and how to recognize and handle them? Please help.

  3. powershell ought to get the credit it deserves for enabling a developer to rapidly create rich output handling complex decisions based upon datasets gathered from various means and implementing a nearly infinite number of actions based on these. Simply put, it can be, and it is, much more than an admin tool, in the right hands.

  4. To start this is a thing of beauty in it's simplicity.
    Does anyone have experience with how much memory the results occupy and doing Get-job | Receive-Job at the end? If I run say a 1000 or 10,000 will this cause memory problems? I am thinking doing Get-Job -State Complete | Receive-Job & then |remove-job inside the loop (and logging it) would reduce the chance of running the host out of memory, or am I just over complicating it?

  5. a) Remoting
    The primary purpose of PS on *nix will be remoting to Win-Hosts, such as Bash on Windows vice versa.

    Due to the nature of *nix as document driven OS, an object based shell does not make that much sense. We're missing the API level. Jeffrey told us so, long ago.

    b) Religious affairs
    It's not about publishing the code (which is nevertheless great!).
    The GPL especially is the denial of the biz model that drives the revenue of Microsoft. So, indeed, haters will hate. Agree.

    But, in for a penny, in for a pound, PoSh is part of Windows which is an expensive, closed down product, increasingly incapacitating the user.

    c) The role of Community
    Sorry to say that, but the PS community is so much more than the few "Get-Expert -wellknown | Get-Random" MVPs. I know it's hard to see that inside the bubble.

    PoSh itself is gorgeous but - at the end - just a shell, such as Korn, C, Z and all the others.

    Far more important: the promise of a datacenter abstraction layer beyond the borders of specific vendors, automation and the refusal of a click UI.

    In this sense, publishing the underlying code is a statement which can't be exaggerated!

    Great Post, Don!

  6. Thanks, Don. This article saved me tons of frustration. I was writing a fairly simple script that would iterate through a list of servers and grab some WMI information. However, using Get-Content, I found that once the file went beyond some threshold, my script would no longer work properly. Implementing your method fixed my problem, and the script works perfectly.

Leave a Reply

Your email address will not be published. Required fields are marked *