Devops is now important in many technology organizations because of two seemingly opposing missions and cultures that need to come together:

• Agile development teams move fast to meet business requirements and implement application changes.
• Operational teams work hard to keep systems performing, ensure computing environments are secure, and manage computing resources.

Agile teams often view operational teams as slow and rigid while system engineers view agile developers as unsupportive of operational needs and reckless when application deployments cause production issues.

These are generalizations, but the two disciplines often have different motivations, terminology, and tools—and this misalignment can create business issues. For example, as startups get bigger, they need to develop operational procedures to ensure stability while minimally impacting their development speed and agility. For large enterprises, they need to find ways to deliver customer-facing applications and internal workflow improvements faster without compromising reliability or falling out of compliance.

Devops aims to address these conflicts with a culture, set of operating principles, and an emerging set of best practices that enable speed of deploying applications and stability running them with fewer conflicts and compromises. This is largely done by providing practices that automate operational steps and standardize configurations:

• For development teams, these practices standardize and automate steps going from developing code to testing, securing, and running applications across multiple environments.
• For operations, the practices drive automation in configuring and deploying infrastructure, monitoring across several domains, and enabling resolution of production issues faster.

Devops practices include:

• Version control and branching strategies.
• Continuous integration and continuous delivery (CI/CD) pipelines.
• Containers that standardize and isolate application runtime environments.
• Infrastructure as code (IAC), which enables scripting the infrastructure layer.
• Monitoring the devops pipelines and health of running applications.

Release management practices in agile development

Devops begins with the practices and tools used for releasing software to compute environments with fundamentals procedures that have been around for decades. They include version control to manage code changes across a team of developers, branching the code base to support different development activities, and version tagging software releases before pushing them into different environments.

The main differences for devops teams is that the tools are easier to use and integrate better with other technologies that automate building and deploying applications. There are also more standardized branching and code merging strategies that are easier to manage with modern version control systems.

For example, many organizations are using Git (including GitHub and BitBucket versions) and other version control tools that offer multiple clients applications, APIs for integration, and command-line tools to manage more frequent or complex procedures. Today, most developers have used at least one version control technology in their projects and so implementing standards is not as hard as it used to be.

Organizations using these tools can adopt branching strategies like Gitflow that standardize branches for production, testing, and development and establish procedures for developing new features or production patches. These branching strategies let teams collaborate on different types of development needs and only introduce code that is tested and deployable into production branches. Teams then use version tagging to label all the versions of source code and other files that are part of a software release.

Automating release management with continuous integration and deployment

Most organizations that require user support after production releases and others that are early in developing their devops practices often follow traditional release-management practices that support constructs like major and minor releases. The more sophisticated teams developing applications that require less user support can practice continuous deployment when there is automation that continuously integrates and delivers code changes to production environments.

To enable more frequent releases, teams look to automate the steps from checking in code to delivering fully tested applications to target computing environments. Continuous integration (CI) is the automation to build and integrate all the software components so that they are in a deployable package. Continuous deployment (CD) tools manage environment specific variables and automate pushing applications to development, test, production, and other computing environments. Together, these tools form the CI/CD pipeline.

For CI/CD to be an efficient automation process, continuous testing must be implemented in the pipeline to ensure that new code isn’t introducing defects and other issues. Unit tests implemented in the continuous integration pipeline ensure that the code committed isn’t breaking any existing unit tests. Other tests that look for code-level security issues and code structure can also be implemented at the integration step. Automated functional and performance that require runtime environments are often automated as part of continuous delivery pipelines.

This automation drives many beneficial behavioral and practice changes that enable teams to make changes more frequently and safer. It drives teams to check in and test code more frequently, which lets defects be found and resolved quicker. Manual deployment procedures are error-prone, which the automation largely eliminates. The automation also takes most of the overhead in pushing new capabilities to users, letting teams deploy more frequently.

Using containers to drive microservices

If CI/CD provides the automation to deliver applications, then containers are the packaging of the application’s operating environment. Developers can specify the operating system, application requirements, and configuration requirements as a container for running the applications in an isolated layer sharing the operating system of its host. Docker and Kubernetes are container technologies that help developers define their application environments in consistent ways.

With CI/CD pipelines to integrate and deploy code and with standardized containers that isolate each application’s computing needs, developers have the tools to manufacture applications services without a lot of overhead. Development teams then have greater options to translate business requirements into microservices that can be deployed, scaled, and leveraged for multiple business needs.

Automating configuring and provisioning with infrastructure as code (IAC)

As automating code integration and delivery and containerizing applications drive application delivery, the next devops practices helps to automate and standardize the infrastructure and cloud services.

Automating and managing infrastructure used to be difficult. Once an architecture was selected, operational engineers went to various infrastructure components to build and configured them according to requirements. The configuration and asset management tools used to capture these architectures required a mix of automated and manual steps and were often out of date or missing critical information. Compute environments were also rigid and, while there were some tools to automate scaling environments, they were often isolated to a specific infrastructure type, required different skills to implement the automation, and had access to only a subset of operational data to determine whether and how to scale.

Today’s cloud environments offer user interfaces that simplify the work for engineers. Engineers can use these tools to set up virtual private networks, configure security groups, and then launch compute, storage, and other required services.

But devops teams take this a step further. Instead of using the web interfaces and manually configuring computing resources, they automate the process with code. Infrastructure as code (IaC) tools let operational engineers script and automate the infrastructure setup and management. The configurations that enable scaling up and down environments can also be embedded in these scripts. Chef, Puppet, Ansible, and Salt are four competing technologies that help implement operational teams implement IaC.

Monitoring devops pipelines and applications

A manufacturing process is only as good as the ability to monitor, alert, and recover from issues. The same is true for monitoring devops and the user experience of running applications and services. As organizations invest in automation, containerizing, standardizing, and deploying applications, a parallel investment in monitoring is a best practice.

Think of monitoring at several levels. At the lowest level is the infrastructure monitoring enabling the recognition and responses when compute resources aren’t healthy or underperforming. Cloud environments today offer capabilities to monitor, alert, and use elastic cloud capabilities to respond to infrastructure issues.

The next layer consists of the tools to monitor and capture metrics around devops automation. These tools become more critical as the number of developers and deployable services increases. These tools provide alerts when builds fail and auditing tools to help diagnose issues.

Last, there are tools that monitor the application uptime, performance, and other runtime metrics. These monitoring tools often test APIs and also perform full browser tests on either single endpoints or multistep transactions. These monitors are a frontline defense to alert devops teams when APIs or applications are operating outside of acceptable service levels.

Where to start with devops practices

There are many devops practices, and they all take time to mature and integrate. There isn’t a prescribed sequence to implementing them or hard recommendations on how much automation to invest in.

Still, organizations should first look to align the culture and mindset around devops principles and then recognize what practices best align with business need. For example, organizations that are already experiencing poor application performance may choose to implement monitoring first to help resolve issues faster and identify root causes easier. Other organizations that are starting cloud migrations may choose to deploy infrastructure as code, while those establishing standard application development architectures may invest in CI/CD pipelines.

Technologists should bear in mind that there is a cost in implementing automation and that not every organization requires continuous deployment. The best practice is to make sure to deliver on business needs first and align devops automation to areas of high repetition where manual efforts are error-prone.