Google Cloud Platform services guide: The right tools for the job

Launched in 2011, Google Cloud Platform (GCP) has had to do some catching up to market leader Amazon Web Services (AWS), its most direct competitor as a purer cloud play. But Google had no experience servicing large enterprise IT, and so has spent several years playing catchup.

Rather than be an AWS clone, GCP has become a unique services outfit that providing massive-scale services, including artificial intelligence and machine learning. GCP’s advantages today include lower pricing via a sustained-usage discount, a much faster network connecting its datacenters, live migration of virtual machines, massive scale and availability zones, and a variety of redundant backups for always-available storage. What GCP doesn’t offer is the wealth of tools and add-ons that AWS does in its bid to address every use case.

GCP has three main services: Google App Engine, Google Compute Engine, and Google Kubernetes Engine:

• App Engine is a platform-as-a-service (PaaS) platform, with which you deploy your code and let the platform do everything else for you. App Engine automatically creates more instances to handle the increased volume for a high-use app.
• Compute Engine is an infrastructure-as-a-service (IaaS) platform, which provides highly customizable virtual machines with the option to deploy your code directly or via containers. Although it requires more configuration and customization, Compute Engine offers more flexibility and costs less than App Engine.
• Kubernetes Engine lets you use fully managed Kubernetes clusters to deploy, manage, and orchestrate containers at scale.

I have compiled a list of the most common uses for the cloud and which Google Cloud Platform components you need for them.

GCP services for devops, software development, and testing

Application development and deployment are the key usage cases for Google Cloud Platform. It starts with Google App Engine (GAE), a managed platform that handles all platform integrations so developers need only to worry about their code. GAE supports Node.js, Java, Ruby, C#, Go, Python, and PHP. Popular devops tools like Chef, Puppet, Ansible, Salt, Docker, Consul, and Swarm are all fully integrated with GCP and enable development for both the cloud and on-premises scenarios.

There is also Jenkins on GKE, a continuous delivery automation server to orchestrate building, testing, and deployment.

Container technology got off the ground with Docker, which AWS supports primarily, but Google is the creator of Kubernetes, a container orchestration platform, that has steamrolled similar offerings like Docker Swarm as the container orchestration platform of choice, to the point even Microsoft and Amazon support it , respectively, on Azure and AWS. It’s rare the industry agrees on anything, but Amazon, Oracle, Microsoft, Pivotal, and VMware are all behind Kubernetes.

Central to this is the Google Kubernetes Engine (GKE), a managed Kubernetes environment that offers additional features such as autoscale, autorepair, and auto-upgrade. Unlike its AWS competitor, Amazon EC2 Container Service (ECS), GKE runs certified open source Kubernetes, so the skills needed are usable in any Kubernetes scenario, whereas Amazon ECS is locked into the AWS platform.

You may choose to use the Docker container engine instead, such as if you have a sizable investment in Docker code already. For those who choose Docker over Kubernetes, there is the Google Container Builder (GCB). In addition to full support for Docker, GCB also gives developers the option of building and testing their code locally so as to not run up a usage bill on the Google Cloud.

GCP services for analytics

Analytics and machine intelligence at web-scale have been core to Google Cloud Platform’s mission since the beginning.

Its big data efforts start with BigQuery, a data warehouse that is free to use with up to 10GB stored and 1TB analyzed per month. It can be used by Google Apps Script, Google Sheets, or any language that can work with its REST API or client libraries.

Cloud Dataflow is used to build data pipelines, either with real time (streaming) or historical (batch) data processing along with ETL processing. Cloud Dataflow also handles massive, multipetabyte data sets and has essentially replaced MapReduce internally for Google. MapReduce is no longer supported by Google, so it encourages MapReduce customers to migrate to Cloud Dataflow, and provides assistance with the process.

Cloud Dataproc is Google’s cloud service for running Hadoop and Spark clusters, which can be integrated with the general GCP service, and the cluster can be resized in real time. Dataproc also allows for custom VMs and has self-repairing VMs.

To get your data ready for processing, Google has Cloud Dataprep. It is serverless, detecting schemas and repairing faulty data or flagging missing data in a structured table.

Finally, if you need to move a massive amount of data to the cloud and uploads would be prohibitively long and expensive, Google offers the Google Transfer Appliance that fits into a standard 19-inch datacenter rack, just like a server. The 2U device comes with 100TB capacity, and the 4U device has 480TB in capacity. You pay the shipping.

GCP services for AI and machine learning

Google Cloud AI is the basis for its machine learning services, with pre-trained models (machine learning APIs) as well as a managed service for more advanced developers and customers to build their own tailored models through the Google Cloud Machine Learning Engine. The Machine Learning Engine is integrated with other Google Cloud Data platform products such as Google Cloud Storage, Google Cloud Dataflow, and Google Cloud Datalab for training models.

In addition, Google recently announced Cloud AutoML, a new set of services to help customers with limited machine learning experience train their own custom models. The neural net-based machine learning service is based on Google’s TensorFlow library for dataflow programming to build models based off images.

Google also offers Dialogflow for end-to-end interactive application development for user interfaces used on websites, mobile applications, messaging platforms, and internet of things (IoT) devices. It lets developers build chatbots and other interfaces capable of natural language conversations with users.

GCP services for compute infrastructure

Google has built a massive infrastructure of highly tuned datacenters designed for scale and minimal response time. This has translated into a massively and highly scalable cloud offering that can process more than 100,000 events per second, or handle tasks with a total aggregate event payload size of more than 100MB/sec.

GCP has more than 100 points of presence across 15 global regions, connected by dedicated fiber optic networks, so you aren’t competing with Netflix traffic for bandwidth. With Virtual Private Cloud (VPC), you can connect your GCP resources or isolate them from one another.

Through the Cloud Delivery Network (CDN), content is distributed through edge points of presence, and the Cloud Load Balancing service can keep one location from getting overloaded by balancing HTTP(S), TCP/SSL, and UDP traffic.

Google’s load-balancing service is the same one used by the search engine that handles up to 1 million queries per second. There is also Google Cloud Dedicated Interconnect for connecting private clouds with GCP; it offers enterprise-grade connections, backed by a 99.99 percent service-level agreement, to GCP with throughput of 10Gbps to 80Gbps.