How intelligent automation changes CI/CD

Today’s thought leaders often say, “All companies are software companies.” This is because a broad range of large and mid-size enterprises have undertaken digital transformation initiatives in the past decade, and these initiatives have had a profound impact on the way these companies develop and deploy software to deliver business value.

In the past, applications were monolithic, deployed on-premises on bare metal or virtual machines, and updates were few, substantial, and infrequent. Today, the new application model leverages microservices, containerization, and continuous delivery, resulting in a high volume of small releases to Kubernetes, VM, and multi-cloud environments. This evolution has enabled new types of business processes and business models, from “as-a-service” everything, to omnichannel 360-degree customer engagement, to business processes driven by real-time IoT data.

Implementing these new software delivery strategies successfully in a cloud-native world requires another transformation — in software development. Organizations must deliver many more software releases with high velocity, frequency, and accuracy without sacrificing software security or ignoring regulatory and business compliance requirements. This evolution increases complexity across the organization.

Geographically distributed teams — in development, operations, devops, security, and compliance — must work faster, more accurately, and with a higher degree of coordination. Similarly, complex, distributed workflows must be highly coordinated to avoid errors and delays while simultaneously increasing the productivity of software delivery team members. This evolution is further complicated by diverse continuous integration and continuous delivery (CI/CD) toolchains, increased security concerns, evolving privacy regulations, and a shortage of qualified technical personnel.

How can organizations increase the productivity of software delivery teams and maximize the business value of their releases in the face of all of these challenges?  First, organizations must adopt internal systems that allow them to evolve their tools and processes over time. Second, they must exercise centralized control over devops, security, and compliance management while granting development teams maximum flexibility when it comes to process and tools. And third, they must intelligently automate the software delivery pipeline.

A new paradigm for continuous delivery

Continuous delivery is the deployment of software changes into production rapidly and safely in a repeatable and sustainable way. This discipline is not new, but a new paradigm for CD is emerging that enables it to serve as the foundation for improving devops management, increasing development productivity, and ensuring governance and security. This new paradigm is based on three principles: open platforms, intelligent automation, and autonomous teams with centralized control.

Open platforms

Open platforms enable seamless integrations with existing CI/CD tools and workflows. This enables enterprises to evolve without gutting their existing toolchains and processes or risking major declines in development productivity, not to mention frustrating developers and management. For example, open integration layers with native APIs for common CI/CD tools can allow development teams to continue using best-of-breed tools.

For maximum productivity gains and control, deep integrations with existing CI/CD tools can provide visibility and control over releases at every stage of the software delivery process, enabling systems to identify risks and control workflows throughout the delivery process. In addition, open integration layers with native APIs for common orchestration tools such as Argo and Spinnaker can allow organizations to change orchestration tools without ripping and replacing their CD platform.

Intelligent automation

Intelligent automation addresses many of the core requirements for successful software delivery. Basic process automation can increase devops productivity by automating routine manual tasks through code. For example, a developer can run a build in Jenkins that then triggers an automated task that pushes the build to Artifactory and kicks off a delivery pipeline. However, combining automation with AI-powered intelligence can turbocharge processes and improve business outcomes.

Intelligent automation can automate routine tasks and then constantly improve automated decision making as the release moves through the delivery lifecycle. Intelligence applied to the release process — when combined with deep tools integrations that provide access not only to events but also to all process data — can automate the detection of software risks and automatically flag release candidates for remediation before they make it to production.

In addition to increased devops productivity and faster and more accurate software releases, intelligent automation provides the means to implement centralized, automated control over compliance and security. By implementing security policies and automation through the software delivery process, companies can implement DevSecOps to make security an integrated part of the development process rather than a review stage at the end of the development process.

Autonomous teams with centralized control

Establishing centralized control is essential for enabling organization-wide devops, security, and compliance teams to ensure compliance, uniformity, and auditability across all software releases. To be successful, centralized control requires a central policy engine that can enforce security, compliance, and business rules at the organization and individual team level. Role-based access control (RBAC) can enable granular permissions for teams and individuals without compromising control.  To enable teams to operate independently, teams must be isolated from other teams in terms of security, deployment targets, and similar factors.

Intelligent automation combined with a policy engine can automate and continuously improve the enforcement of security and compliance policies, reducing demands on the devops, security, and compliance teams. To further enhance performance across software development pipelines, organization-wide best practices and reusable deployment patterns can be implemented to improve the productivity and accuracy of development teams.

Continuous delivery evolves

These new foundational elements of continuous delivery are now beginning to make their way into products in the following ways.

Open source projects

Open source CI and CD projects continue to evolve. For example, to meet evolving security concerns, Spinnaker, an open source, cloud-native CD solution, now includes multiple options for authentication (identity management) and authorization (access management). In support of centralized control, Spinnaker has also taken a smart approach toward these key security capabilities. Instead of coding a new and proprietary login solution, Spinnaker takes advantage of modern security protocols, including OAuth 2.0, SAML and LDAP, enabling Spinnaker to integrate with the identity and access management solutions already in use at most organizations. Spinnaker also integrates with common authorization solutions, such as Google Groups, GitHub Teams, SAML Roles, and LDAP groups.

Basic process automation

Automation of routine processes is becoming common in commercial solutions, including both proprietary solutions and products based on open source CD solutions. For example, commercial solutions such as CodeFresh, Armory, and OpsMx are all built on open source CD projects, such as Argo and Spinnaker. Proprietary commercial solutions are also available from companies such as Digital.ai, Harness, and Broadcom.

Basic artificial intelligence and machine learning

AI and machine learning are emerging within commercial solutions such as Harness and OpsMx. One example of this is a machine learning-powered continuous verification process that learns from previous deployments and creates a baseline of what is a good deployment, which enables anomaly detection. Likewise, CI/CD tools from New Relic, Datadog, Honeycomb, and Splunk use AI/ML to provide additional insights into software performance and quality. 

Intelligent automation

Intelligent automation combines AI and robotic process automation (RPA) technologies to streamline and scale processes and decision making across organizations. Intelligent automation is emerging within a limited number of commercial CD solutions such as OpsMx, allowing organizations to go beyond automating routine processes and workflows.

Intelligent automation enables risk analysis of software releases, automatically determining whether a release meets the criteria to pass to the next pipeline stage without an unacceptable level of risk that it will fail in production. This level of intelligence can also automate policy compliance, ensuring all governance rules and best practices are followed. Industry-leading companies go further and combine intelligent automation with advanced deployment strategies such as blue-green, canary, and progressive deliveries, to deploy software faster and with lower risk than ever before possible.

It is essential that software delivery processes keep pace with the requirements of digital transformation at the organizational level. Failure to do so will lead to software delivery challenges that result in slow releases, high release error rates, security and compliance failures, and user and customer frustration.

The good news is that devops can implement a new foundation for its CD processes today to ensure faster and higher quality software releases. As intelligent automation capabilities become more widely available, devops can position itself as a prime driver of digital transformation acceleration, delivering new software capabilities faster, more frequently, and more safely in ever tighter timeframes.

Gopal Dommety is the CEO of OpsMx, provider of an intelligent continuous delivery platform. Gopal is a serial entrepreneur and technology visionary.

—

New Tech Forum provides a venue to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to newtechforum@infoworld.com.