Using Pulumi 3.0 to manage Azure infrastructure

The Pulumi IaC (infrastructure-as-code) platform takes an interesting route to delivering cloud infrastructures. Instead of using a domain-specific language like Bicep or declarative configurations like Azure’s ARM templates, it offers a way to use familiar, general-purpose computing languages to describe and programmatically construct infrastructures, using an open source platform.

You write code in your choice of editors, using standard language tools to handle code formatting and linting, and it’s run against the Pulumi engine, which delivers virtual infrastructures and configures service endpoints ready for your code. As Pulumi files are just standard C# or Python or JavaScript, or even Go, they can be managed alongside your application code and included as part of any IDE project.

Build infrastructure in your code editor

It’s a powerful approach; you don’t need to learn any new languages and you can take advantage of control flows in a way that a declarative, domain-specific language can’t support. As a result, a single piece of Pulumi code can define all the resources that are used by any project from your organization. It’s how you call the code that matters, using flow control elements to deploy groups of servers and set up software-defined networking.

The same unit test tools you use for your code can be used for infrastructure, running code to see what would be deployed if you were in a live environment. By testing infrastructure definitions and code before going live, you’re able to avoid misconfigurations and errors. Deploying a small application on a large Azure server could be a costly mistake, one that would be easy to overlook without hands-on operations support.

Pulumi offers tools for architects and operations teams to apply architectural and policy standards to infrastructure. You can set limits on the servers being provisioned or the levels of service being used, as well as define different characteristics for development, test, and production. Using architectural standards like this puts guardrails around your Azure deployments in the same way Visual Studio can enforce coding standards.

Native support for Azure

The initial release of Pulumi was a cross-cloud tool. Code written in Pulumi could deliver infrastructure on any cloud provider’s platform. But that limited you to using only the services that could run on all clouds; you couldn’t use it to build Azure-specific applications that relied on Azure services and APIs.

Over the past few releases, Pulumi has started introducing support for what it’s calling native providers, with Azure the first to go into general availability as part of the recently released Pulumi 3.0. Microsoft has been investing heavily in documenting and adding metadata to its APIs, with Azure providing data for its REST APIs. This allows Pulumi to generate a new version of its Azure native provider nightly to keep up with new features and updates.

This doesn’t mean that the APIs and calls you use need to change at the same rate. Pulumi’s tools support all versions of an Azure API, so your code and infrastructure don’t need to change until Azure deprecates the version you’re currently using. If a new service arrives that you want to use in your applications, it will be available as soon as Azure publishes the API. The same API automatically generates documentation, with new versions for each supported language.

It’s a useful tool, and Pulumi clearly is proud of what it can do. As the company notes, it’s made 279 different builds of the native provider, adding 166 new Azure infrastructure APIs.

Infrastructure budgets as code

Those APIs include governance and billing APIs, so you can wrap controls around your infrastructure to manage spend and security. Infrastructure code can include consumption budgets to help you track costs and tie resources to specific accounts. Budgets can be measured per month, per quarter, and per year. Budgets can also be usage based, with notifications automatically sent to email addresses when quotas or thresholds are exceeded.

Managing costs on Azure can be complex, and this approach should help you set appropriate safeguards for infrastructure and service usage. Not everything will be covered by this tool, but you should be able to get an idea of what your services are costing you and how you can spend more effectively, perhaps by reprovisioning on lower-specification servers.

Automating infrastructure with code

One of the more interesting features in Pulumi 3.0 is its Automation API. This allows you to embed Pulumi functions inside your own applications as a tool for managing cloud infrastructures. It’s a fascinating approach that’s flexible enough to be used in many different ways.

One obvious option is building it into an internal portal for self-service infrastructure deployments. Instead of giving all your engineers access to the Azure portal, an internal service can be populated with approved server images and other infrastructure elements. Engineers needing a temporary test or development environment can pick and choose the tools they want and automatically deploy them without requiring operations support. The service can recover resources when a test is complete, keeping billing to a minimum.

Alternatively, a set of Pulumi scripts for an application-specific infrastructure can be kept in your choice of repositories. An application build in Azure Devops would not only compile, build, and package code, but a pipeline action could be used to trigger the Pulumi Automation API and deploy an Azure infrastructure ready for the code.

Having an API for infrastructure management allows you to start writing your own management code, consuming data from Azure’s management APIs to use in conjunction with tools such as the Cognitive Service prediction APIs to proactively deploy additional application infrastructure elements in response to changes in load or signs of failure in a currently running instance. By deploying infrastructure only when it’s needed, you can keep costs to a minimum while ensuring your applications remain responsive and stable.

The idea of building code that can manage itself is attractive, especially in organizations where operations resources are hard to come by. Self-managing applications will need to be tested carefully before being deployed, and you’ll need to have a good devops culture to use these techniques successfully, with applications operations staff able to monitor an entire suite of applications and tools.

This is a major release of a tool that keeps getting more and more useful. Building dynamically on top of Azure’s ARM APIs ensures your infrastructure can always use the latest services. Your applications can build on them as soon as they are released, while still taking advantage of a hands-off, automated infrastructure using your choice of languages. Pulumi is working on adding new languages to its toolbox, too, with Ruby and Java currently under development.