Build your own private cloud

The primary mission of computing has always been to automate business. The secondary mission has been to automate the automation, a quest that grows ever more urgent as data center technology achieves such towering complexity it threatens to collapse under its own weight. 

The private cloud is the latest attempt to boost the efficiency and agility of the vast assortment of hardware and software sprawled across data centers. The idea is to pool compute, storage, and network resources and manage them from a central software control point rather than scrambling to provision, monitor, manage, and reconfigure all that infrastructure manually.

Predictably, the organizations with the most experience designing and running shared infrastructure are the public cloud service providers themselves. Amazon, Google, Microsoft, and Salesforce can’t manage their data centers the old-fashioned way — with static chunks of infrastructure dedicated to individual application instances — and hope to deliver services to millions of subscribers over the Internet reliably and cost-effectively. These pioneers’ experiences running public clouds at scale have yielded the basic models for the private cloud.

Downsizing the cloud

Some are quick to point out that public cloud providers typically offer a few services at massive scale, not the wide diversity of applications at the small or moderate scale characteristic of the typical enterprise. So is the public cloud model applicable to enterprises at all? Isn’t “private cloud” just a marketing phrase for virtualization plus a little bit of automation?

Actually, wherever a sizable chunk of server virtualization is in effect, the public cloud model may apply, even if the scale and the details of the implementation may not. Virtualization has become so widespread, we tend to think of pooled resources, particularly VMs spread across a farm of physical hosts, as simply a fact of life. In truth, this consolidation represents a profound transformation in the way we manage computing infrastructure.

That difference emerges in sharp relief when virtualization reaches critical mass. At a certain point, such as reaching hundreds of physical hosts and thousands of VMs, or serving numerous disparate “tenants” who demand instant gratification, managing virtualization becomes a qualitatively different proposition.

For one thing, you need private cloud software to allocate resources and costs back to line-of-business stakeholders. When possible, you want to take advantage of the agility of a virtualized infrastructure by allowing those stakeholders to provision their own resources. Plus, as you converge data and storage networks, you’ll want to virtualize storage and network resources along with servers.

The software to accomplish all that and more has already been released in some form by Eucalyptus, VMware, OpenStack, Microsoft, Citrix, and others. These various private cloud software stacks, however, are not intended to replace every lick of infrastructure software in the data center. They provide a framework that interoperates with much of the software already in place and helps admins wrap their arms around the totality of virtualized data center infrastructure.

Elements of the private cloud

The elements of the private cloud form a new management layer across a wide swath of data center functions. It sounds like a tall order, but keep in mind moving to the private cloud needn’t be an all-or-nothing affair. It can be rolled out incrementally. Although a number of hardware and software vendors imply that you need racks of new hardware and an industrial-size barrel of software spaghetti to build a real private cloud, you can typically graft individual cloud features one by one onto an existing virtualization infrastructure.

Compute. At the most basic level, in a private cloud, the compute component does pretty much what virtualization management software does: provision, deprovision, start, stop, suspend, clone, and migrate VMs from a central control point. The main difference is that the intent of the private cloud is to enable you to manage across as much virtual server infrastructure as possible — which in many cases means support for multiple hypervisors.

Storage. Storage management across platforms has been a dream for more than a decade. Most private cloud platforms strive to support object storage and replication, along with block storage and snapshot management, across diverse data center storage systems, although this is still a work in progress. Another essential part of private cloud storage is efficient management of virtual server and virtual disk images.

Networking. Here, private cloud software extends into the cutting-edge realm of SDN (software-defined networking). You can create virtual ports, attach VMs, configure virtual switches, and so on. You can provision private networks within the greater network infrastructure, an essential capability when you’re creating a large private cloud to which several different groups of users will have access. In a modern data center with lots of 10G switching and converged data and storage networks, the ability to pool and carve up bandwidth dynamically can be especially powerful.

Metering and chargeback/showback. When infrastructure is shared, you need a way to divide and meter pooled compute, storage, and networking resources and automatically charge line-of-business accounts — or at least show those charges to the appropriate stakeholders. Without that capability, IT people either get mired in spreadsheets estimating charges or settle for inaccurate or unfair cost allocations that will come back to haunt them.

Self service. Perhaps the greatest of all efficiencies offered by the private cloud is the ability to tell stakeholders: Here, you do it. Developers, for example, can configure and reconfigure their own dev and test environments using a self-service portal. Business units can even “order” and install applications, which are automatically provisioned and made available without IT necessarily being involved. Naturally, configuring the portal, policy, and automation magic that makes it all work requires time and effort. Also, the standard choices for stakeholders will necessarily be limited. But as the menu expands, the time not spent dealing with requirements, provisioning, configuring, and so on may represent a huge savings.

Automation and orchestration. For the magic self-service portal to work, you need automation, or basically, a library of scripts that perform common tasks: server provisioning, shared storage setup, network settings for a VLAN, and so on. Orchestration assembles those automated tasks into predefined workflows for specific applications or services, which may require signoffs by gatekeepers. For example, if a developer wants to test a massive application that will require a large chunk of resources and a copy of a live database, the workflow software would automatically fire off an approval request to management.

Application management and monitoring. Private clouds may extend all the way to managing the application layer. This includes templates for deploying and configuring applications, but also monitoring and reporting on application performance to meet internal SLA requirements, with dashboards to supply admins with the insight they need to deliver on those commitments. At the most advanced level, monitoring may trigger autoscaling so that performance levels can be maintained without human intervention.

Security. The need to isolate multiple tenants in a shared infrastructure makes security both critically important and dauntingly complex. An identity management system that provides multiple levels of authorization according to user group assignments is a key component. It also should be capable of extending granular access controls to external users (such as customers and partners) as well as internal groups.

Which elements of the private cloud you implement first will depend on the workloads you have in mind. If you plan to host partners in your private cloud, the security and network components required to support virtual private clouds could well be the first order of business. If managing resource allocations for internal dev and test is a top priority, a portal for developers to self-provision (plus the automation and orchestration behind it) might be first in the queue.

On the other hand, it goes without saying that deploying a full-featured private cloud in an environment where business units do not have the scale, interest, or ability to take advantage of its self-service capabilities would be a massive waste of time and money.

Many large and medium businesses can benefit from the power of the private cloud. The really hard part is getting everyone to agree on the rules, rates, and policies for the automated systems. If you can clear that hurdle, then it’s a matter of virtualizing everything you can, prioritizing workloads, and gradually adding more private cloud functionality as you work your way up the stack.

This article, “Build your own private cloud,” was originally published at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.