Azure Copilot: An AI assistant for Azure ops and troubleshooting

Microsoft has been polishing up its AI-powered Copilot in Azure for months now, and finally decided it’s ready for everyone to use. A public preview of Copilot in Azure will roll out relatively quickly, over a couple of weeks. If you’re not able to access Copilot in Azure immediately, rest assured you should see it in your Azure Portal soon, where it can help you manage, secure, and tune your Azure cloud infrastructure.

I talked with Erin Chapple, Microsoft CVP, Azure Core Product and Design, about the new service and where it’s likely to go in the future. Like Microsoft’s other service-oriented Copilots, Copilot in Azure isn’t the general-purpose Copilot you’ll find in Bing. Instead, think of Copilot in Azure as a natural language interface to Azure services that can work directly with the Azure Resource Graph and Azure APIs, while retaining context between queries.

[ Download our editors’ PDF AI as a service (AIaaS) enterprise buyer’s guide today! ]

It’s important to understand that Copilot in Azure is limited to helping you with Azure services running in your tenant. So, if you want to ask Copilot about a security issue, for example, such as what IP addresses are being used to probe your login service, you need to have both Azure Defender running and a Security Center instance to expose the relevant information to Copilot’s large language model (LLM).

In other words, you can’t use the complex retrieval-augmented generation that powers Copilot in Azure without the appropriate data sources. And while much of the service builds on Azure Resource Manager and Azure Resource Graph, there are features that require more information about your tenant.

How Copilot in Azure can help

Other options offer new ways to solve perennial problems. The question “Why is my subscription costing so much?” can help track down misconfigurations or instantiated VMs that haven’t been shut down. By having access to the data that underpins your tenant, and using lessons from across the entire global platform, Copilot in Azure is able to point out issues that you may have missed. You can then use it to tune existing apps and services or help you choose a better set of VMs for your application.

That last option is one that Chapple is excited to see being used. “We’re using insights from the Azure App Services from a diagnostic standpoint and exposing that, and I think that is building out more and more capabilities,” she told me. Getting that information from the Azure Portal requires bringing together information from different tools, using Copilot as a bridge, collating and sorting the information you need to get the best performance from your code.

As Azure encompasses many different services, Copilot in Azure is building out a library of skills that spans both administration and development. For example, if you’re using Copilot with Azure Kubernetes Service, there’s a skill to create and manage backups, and another that helps you find the YAML configuration files you need to tune your applications. Copilot can even assist you when you’re writing kubectl command lines. Copilot learns from the existing library of Azure design patterns and practices, with Chapple pointing out that “the model is trained over that, and it can actually use that in its response in order to guide you down the right path from the beginning.”

Other Copilot skills help you work with Azure-hosted data, adding the ability to go straight from natural language to SQL when working with your Azure SQL databases. All you need to do is ask questions about your data and Copilot in Azure will generate the appropriate T-SQL statements for you, ready for use in code or in your database administration client. Copilot will explain the queries it generates, making it a useful learning aid, as you will be able to use its examples as lessons to help with future database queries.

Also keep in mind that Copilot in Azure is not a one-shot tool. You can use it to generate reusable assets, for example Azure Resource Manager templates or Azure CLI scripts. After all, reusability has been at the heart of Microsoft’s administration tools since the launch of PowerShell, where it’s always been possible to see, save, and edit the scripts used under the hood of a GUI. Copilot works much the same way, only for natural language interactions, delivering output you can check, share, and reuse.

Troubleshooting Azure with AI

Copilot in Azure is intended to help troubleshoot problems, building on data from Azure’s support library and providing an interactive, conversational path that can narrow down to a specific solution. You don’t have to wait on a round-trip email conversation, or for a phone call. Instead you work with the Copilot chatbot, which allows you to start with a proposed solution and then refine your questions to get the answer you need. Again, getting sample scripts and templates will help you avoid similar issues in the future.

As Chapple notes, “It’s a great educational tool in many ways to say, ‘What are the settings that I can use to restrict inbound connectivity?’ as an example.” Her team has been working with early adopters to understand their use cases, with another key use being helping administrators understand their growing cloud environments. Here they’re asking questions about the environment as the sprawl starts to grow, she says, and questions like “What VMs in this resource group have restricted outbound connectivity?”

While Copilot in Azure is intended for all Azure users, the AI assistant does have a focus on administrative tasks. That doesn’t mean developers can’t use Copilot to easily retrieve the Azure CLI commands needed to enable some new infrastructure. But Copilot in Azure is intended mainly to ease running large Azure infrastructures, which remains an issue for many organizations.

Chapple expects the role for Copilot in Azure to expand, and even take it outside the Azure Portal into other applications and tools. She describes Copilot as a way of delivering to users the information Microsoft has, through many different channels. “It won’t just be the Azure Portal, we want to expose it through the mobile app, through the command line, and through other experiences. So that customers can have this companion right through that operational phase from design to deploy to troubleshoot, to gain access to the information that we have.”

There’s a possible future for all of Microsoft’s many Copilots here—as a family of intelligent agents that could hand off context between each other. Imagine starting a query in Copilot in Azure and ending up in GitHub Copilot, without losing your initial question. If that sounds like how we talk to our colleagues, well, that’s just as it should be—a collaboration between you and your machine assistants.