Understanding programmatic Azure infrastructures

The more Azure grows, the harder it is to manage through its portal. After all, graphical user interfaces rarely scale beyond a few machines. With a global Azure deployment, you’re going to be running hundreds, if not thousands, of virtual machines, containers, and Azure service endpoints. It’s not a job for fallible humans, where one misconfiguration can result in a significant service outage.

That’s where Azure’s Resource Manager (ARM) comes in. It is an important part of work with Azure. The infrastructure elements that make up an application also make up a template that’s stored, ready for use and reuse. Although you can store them in your Azure account, you can able store templates outside Azure, such as in a GitHub repository. Because they’re JSON documents, you can edit and manage them in any tool that can work with JSON, like Visual Studio Code or the full Visual Studio IDE.

Azure Resource Manager templates and the Azure CLI

Templates are easy to manage and deploy with the Azure Portal, but things get much more interesting when you start to mix in the Azure CLI. With the CLI running locally on your PC, managing Azure through a command line makes a lot of sense once you’re working at scale.

Instead of clicking through a GUI, a deployment is now a matter of a couple of lines of script. You first set up an Azure Resource Group, then you deploy your templates. Once provisioned, you get a notification in the CLI and your application is ready to run. Note: If you prefer to use the Azure Portal’s built-on Cloud Shell, make sure your templates are in a storage account associated with the account that’s running the shell.

Although you can deploy on as many as five regions from the same command, you’ll likely want to automate deployment across many more regions if you’re implementing both regional and global reliability for your apps. To handle this (and to support deployment to separate development, test, and QA environments), use a separate JSON file to handle deployment parameters for your template.

Once you’ve built the Azure CLI commands and Azure Resource Manager templates for your deployment, you can wrap them in scripts to simplify operations. Azure CLI runs inside Bash on Unix or in PowerShell on Windows. This gives you the flexibility to build and trigger deployment scripts in your standard environment, and to automate deployments as part of builds or from monitoring tools, adding new resources as necessary.

Code your infrastructure with Pulumi

An alternative, and very interesting, option for scripting your deployments comes from Pulumi. Founded by a group of former Microsoft, Amazon, and Chef engineers, Pulumi is building a cross-cloud tool that uses code to build and deploy infrastructures. There’s no need to learn a domain-specific language for each environment you want to use: Although Pulumi might push an infrastructure using Azure Resource Manager or a similar templating language, you’re coding it in Python or JavaScript.

By building on top of code, you can quickly see what’s repeatable and what can be built into reusable packages. Instead of defining a new server VM each time you want to deploy a slightly different version, you can have a library of modifiable infrastructure blocks that can be customized at runtime, using variables and other familiar constructs to change storage, memory, and CPU as you want. Similarly, you can use code to work with platform services, and with newer serverless systems.

Tools like Pulumi will also work at a higher level; for example, configuring and deploying Kubernetes. Here, the platform handles creating the YAML used by Kubernetes, though you don’t need to know the specific structure of your Kubernetes control files. All you need to do is write code that defines the containers you’re deploying, and how many instances you’re deploying. Your deployment becomes a function, and your containers are constants. Everything is handled by a set of libraries that Pulumi manages, though you can apply your own customizations for any specific features in your Azure environment.

Deploying multicloud serverless applications

Pulumi also speeds the creation and deployment of serverless applications, with code that defines APIs and the building blocks you’ll use to build your serverless code. Blocks map across the three main public cloud platforms, so you can compose serverless applications either by using a high-level cross-cloud set of libraries or focusing on Azure-specific implementations. You don’t get the same level of control with the high-level tools as you do with the service specific libraries, but it’s a trade-off that’s worth considering.

Using Pulumi to build functions makes a lot of sense, because application code and deployment code can be built together. You can create a database in the same code that uses the database to store the results from an IoT device, because that database is an object in its own right. The code that runs your serverless application is another JavaScript function.

There’s a lot to like in this approach, from using code to build complex Kubernetes infrastructures (and to keep track of your distributed microservice architectures) to having a series of commands that act as a library of all your key server images, with code to build the underlying network. Another advantage comes from coding against a deployment engine: If you’ve already deployed an application and you make a change to the deployment code, Pulumi will push only the changes rather than deploying the entire application again.

Living the devops dream

Way back when, before the public cloud and virtualized infrastructures, you had to wait months to get access to physical hardware to test and to deploy our code. By making infrastructure code, you can now deploy application and infrastructure as an idempotent entity, unchanging until you push an update by deploying an entire new service. Using tools like these, and with low-cost developer Azure subscriptions, you can use the same approach to quickly create and test full-scale environments, tearing them down as soon as you’re done.

Infrastructure as code has long been one of the dreams of devops, adding configuration and deployment rules to continuous deployment systems and their build pipelines. Once you’ve created a build script for your application’s infrastructure, it can be stored and managed in a source-control system, as well as treated as an endpoint for builds, deploying to local simulators, to test Azure subscriptions, and to global production.