Google adds Gemini to BigQuery, Looker to help with data engineering

Google Cloud is adding its generative AI-based chatbot Gemini into its fully-managed data analytics service BigQuery in order to ease several data-related tasks for enterprise professionals, it said Thursday.

Gemini inside BigQuery will aid with code generation, code completion, code explanation (SQL, Python), help with data canvas, and provide partitioning and clustering recommendations, said Gerrit Kazmaier, vice president of data analytics at Google Cloud.

Analysts said they expect data professionals to see the addition of Gemini as a welcome move, as data engineering is often the most time-consuming task in many enterprise data and analytic processes.

“The generative AI-based chatbot can help data professionals by augmenting various tasks, such as assisting with data imputation to data model discovery and maintenance,” said Bradley Shimmin, chief analyst with Omdia.

Explaining the importance of data partitioning recommendations from Gemini in BigQuery, Nucleus Research senior analyst Alexander Wurm said that these recommendations optimize query performance, which can alleviate “costly” technical challenges associated with big data analysis.

Rival data analytics platforms or service providers are already putting generative AI into their offerings — or planning to do so: Microsoft has put Copilot into Fabric and AWS is putting its  Amazon Q into multiple data analytics services including Amazon Kinesis and Glue.

For Steven Dickens, chief technology advisor at The Futurum Group, “The introduction of Gemini in BigQuery adds competitive pressure on rival data analytics platform providers as this is a dynamically competitive space where each provider continuously is seeking to outdo the other by offering advanced functionalities.”

Other alternatives from the likes of Oracle, MongoDB, Databricks, and Snowflake also offer similar solutions, Dickens added.

However, Wurm pointed out that since all major data analytics provider are focusing on the strategy of simplifying user experiences via generative AI, the competition is no longer about who offers generative AI, but rather, which vendor can provide a pricing model that will reduce friction to enterprise adoption and generate the greatest return on investment.  

Other updates to BigQuery

Additional updates to BigQuery include the general availability of the Delta format support. In 2022, Google had added support for Apache Iceberg.

Google is also adding support to analyze structured, unstructured, and open-format data using SQL, Spark and Vertex AI integrations.

These integrations, according to Shimmin, point towards Google trying to follow the trend of enterprises adopting a data fabric, a unified API layer to access various data sources and types, to link up existing isolated datasets inside an enterprise.

Google has also added support for Apache Spark and Apache Kafka to BigQuery.

While the support for Kafka will allow data professionals to more readily incorporate real-time data into their analytic products, the support for Spark will allow data professionals to scale their operations by running streams in parallel, Shimmin explained.

However, David Menninger, ISG’s executive director, pointed out that adding support for Kafka and Spark were far from a novelty and said that “nearly all data platform providers support Spark and Kafka.”

Enterprise these days, according to Shimmin, differentiate vendors or providers on the basis of how well they implement these integrations and how well the database interweaves with the vendor’s broader offerings, for example, Vertex AI and Looker in Google’s case.

Looker gets Gemini, but only in preview

Google has also added Gemini its to business intelligence (BI) tool Looker, but only in preview, the company said, adding that the generative AI-based chatbot could help with formula assistance, create metrics from complex formulas, slide generation, and new ways to showcase data.

“The addition of Gemini to Looker potentially lowers the technical barrier for data analysis, enabling more users to derive insights from data,” said Dickens from The Futurum Group.

Omdia’s Shimmin pointed out that the addition of Gemini to Looker shows that Google is following a broader market strategy employed by rivals such as Microsoft and AWS to incorporate more generative AI functionalities into business intelligence products such as QuickSight and PowerBI.

“If the implementation of the generative AI chatbot is done right, these can greatly accelerate existing workflows among data professionals,” Shimmin added.