Introducing CloudEvents for event-driven development

A recurring argument about cloud application development suggests that “it’s someone else’s computer.” While there’s a kernel of truth in there, especially if you’re only lifting and shifting code from an on-premises datacenter to an IaaS pubic cloud, it’s an aphorism that ignores the economic benefits that come from adopting newer, cloud-first development models. Well-designed cloud apps can process hundreds of thousands of transactions for a few cents, operating costs that are hard to match even in the best run of on-premises datacenters.

Microsoft has been investing heavily in serverless computing models, with the new Service Fabric Mesh and its Azure Functions event-driven programming platform. They’re key elements of any new cloud development, especially if you need to respond to events sourced from other applications. Azure also includes tools for managing those events, and I’ve recently written about Event Grid, the publish-and-subscribe framework that Microsoft uses to marshal and direct events to serverless microservices running as Azure Functions (or to its serverless-hosted low-code Azure Logic Apps and Azure Flows).

It’s easy for Microsoft to source events from its own first-party services. Drop a picture in Azure Blob Storage, and an Azure Function can fire and process the image through Microsoft Cognitive Services or any other API. But things get harder when you’re working with third-party event sources, either in your own code or from other applications and services. You could use a webhook to deliver an event, but that requires code that can handle asynchronous callbacks, with compute resources required to monitor the callback URL for events.

The challenge is to find a common way of describing and delivering event information. If you can deliver that, you no longer need to learn a new way of working with each new event provider that your code needs to use. Instead of custom code, you can use common libraries, and your code becomes more portable and more resilient, making it easier to switch to new event providers in the event of changes.

Introducing CloudEvents

A cross-industry group—including Microsoft, Red Hat, Serverless, Google, and Amazon—has been working on solving this problem for some time, coming up with a standard for event data called CloudEvents. As one of the leads on the project,Microsoft has been quick to adopt CloudEventsin its own tools, with support in Azure Event Grid and Azure Functions.

With so many big cloud providers involved, CloudEvents needs an effective governance framework to deliver the solution developers need. So, it’s good to see that it has been adopted by the serverless working group of the Cloud Native Computing Foundation, the home of projects like Kubernetes and Helm. Much of the software infrastructure behind Azure is open source, so working with the CNCF ensures that there’s an open design process that can support multiple, cross-platform implementations of the standard. That’s an essential requirement for a platform like Azure, as if it’s to be a successful public cloud it needs it must be able to support cloud-to-cloud and service-to-cloud events.

At the heart of CloudEvents is a JSON schema that describes any event. It contains details of the event type, its source, an ID, and time. Most important, it has a data block made up of user-defined data. That’s where you’ll define and deliver the data that’s sent to the application consuming the event. If you’re developing a CloudEvent intended for third-parties, you can also include a URL pointing to a schema for the data payload, making it easier for services like Azure Event Grid to handle conversions between different event types. An active GitHub repository makes it easy to get involved with the development process.

Microsoft’s support for CloudEvents is currently in beta, and only in a small subset of its Azure regions. You’ll need to explicitly enable the Event Grid extension in the Azure CLI, and then when you create the topic for your event youset its input schema to use CloudEvents. Similarly, you can ensure Event Grid emits Cloudevents-format data, by setting the appropriate delivery schema. By tweaking input and delivery schema support, you can use EventGrid to translate between event types; for example, delivering all incoming events to your Azure Functions as CloudEvents.

It’s early days for CloudEvents, so if you do start using it, you’ll need to monitor your event deliveries carefully. With a common schema for event messages, serverless applications can now operate across clouds, taking advantage of the portability offered by Azure Functions container-based deployment option.

CloudEvents and gRPC: a cloud API combo to wish for

There’s another aspect to the arrival of open standards like CloudEvents: Developing good APIs is hard. RESTful services are hard to compose, GraphQL is hard to version; and althoughgRPC is developing nicely it needs more support before it gains wide adoption. With both gRPC and CloudEvents under the auspices of the CNCF, there’s the prospect for the two to become the basis of a new set of API standards for cloud services and applications.

But they serve different purposes: gRPC is, as it says in its name, a tool for remote procedure calls over HTTP/2, while CloudEvents delivers notifications of state change. In a cloud-hosted serverless context, you use CloudEvents to trigger a serverless function, with results being delivered to back-end services via gRPC. Alternatively, you could make calls to a service via gRPC, saving results to a storage account or a database, and then triggering the next service with a CloudEvent.

Between the two, you have all the tools you need to build an effective set of APIs for a service, both as inputs and as outputs.

With serverless computing a key component in modern cloud architectures, getting this right is increasingly important. An effective set of standards for handling events and API calls will make it easier for developers to extend existing services to support serverless endpoints. Similarly, they should also reduce the amount of work needed to build even support into serverless platforms. Instead of having to write custom event handler code, you’ll be able to import a schema definition into an IDE, write the code needed to process the event data, build, and deploy.