Cloud database and cloud infrastructure people need to talk

Most development and deployment teams now work better than they did prior to cloud computing. The rise of agile and devops/devsecops set the framework for ops and dev teams to work more closely together. Dev now works well with ops, and the other way around.

However, I’m noticing some flaws in the system. Namely, the cloud database designers who work with cloud-native and non-native databases don’t seem to be syncing up with the infrastructure designers/architects who provide important resources for a cloud database, including storage and compute.

This results in a few bad outcomes.

First, databases run out of resources while they do core business processing. In many instances, it gets fixed after a warning message shows up on somebody’s console, but it still shuts down the databases for a time, and those important business transactions go bye-bye.

Second, is an outright outage. The database runs out of resources during processing and just stops working without warning. This could even corrupt the database, which means you’re only as good as your last backup.

The issue is that all cloud-hosted databases are not equal, but the cloud infrastructure engineers think they are. Some databases can auto-allocate their own resources and only require that somebody pay the bill at the end of the month (serverless). Most require that those charged with infrastructure have some knowledge of the database, including the number, amount, and type of storage required; database growth; use of a data cache; CPU type and size requirements; etc.

If you think this sounds a lot like configuring and sizing servers for on-premises databases, you’re right. Many of these popular, traditional databases still don’t know if they run on-premises or in the public cloud. The process of sizing systems for specific databases and usage assumptions are pretty much the same.

We know how to solve this problem. The trouble arises when it’s time to actually do it. Communications often break down between those who design, configure, and deploy a cloud-based database, and those with the keys to the resources the database requires. Avoidable database outages happen more than they should, and that negatively impacts the business.

The fix is easy. Include a step in the development and operations process where the people who will select and deploy a particular database communicate detailed requirements to the people who will allocate, size, and configure any resources that the specific database may need. This is also the time to address security, governance, and management of the database, and that requires talking to still another group within the company.

Here’s my solution (sorry, database people): The database admins and infrastructure admins believe that communications should be symbiotic between the two parties. However, those charged with implementing the database need to pay more attention to infrastructure, and they need to be more proactive.

The days of overbuying processing power and storage in a data center are over. It’s a balancing act to bring in just the resources you need and to do so while never running out of resources. This falls into the laps of the database people, with the support of cloud infrastructure people.

Database people need to become much more educated about the inner workings of their public cloud hosts. The public cloud and on-premises databases didn’t mix when the on-premises database sat down the hall. Now we need to mix them.