Data infrastructure company Databricks says its new platform helps users create secure, tailored, enterprise-specific apps in just minutes.
Every enterprise today is looking to get as much as possible out of its data and AI investments. But when building in-house apps, developers can struggle with infrastructure constraints and maintenance, security, data governance, compliance, and other issues.
“Companies want to build custom experiences,” Shanku Niyogi, VP of product at Databricks, told InfoWorld. However, “as soon as you start to build a custom application, you start to fall off a cliff.”
Databricks says its new platform, Databricks Apps, can help deal with these complexities. Today available in public preview on AWS and Azure, it says the platform allows users to create secure, tailored, enterprise-specific apps in just minutes.
“Applications ultimately enable customers to really get value from their data and all their AI investments,” said Niyogi.
Easy to build, deploy on an open, secure network
Internal data applications have a variety of challenges, Hyoun Park, CEO and chief analyst at Amalgam Insights, told InfoWorld. Building data governance and controls is always a “major effort,” and apps need to be written in a language and with frameworks that can be supported on an ongoing basis. Companies need to worry about servers and cloud computing resources, and also must determine how to choose the right model for each use case while supporting customization, prompt engineering, and model augmentation.
“Model flexibility has become an increasingly challenging aspect of data apps, especially for companies that have traditionally only built data apps to support traditional analytics and reporting use cases,” Park explained.
In contrast, Databricks Apps is easy to build and deploy, and has an “open approach,” with Python as its primary language, Niyogi explained. If users know Python, they can build an app in as few as 5 minutes, he said.
The platform provides automated serverless compute, meaning users don’t need IT teams to set up infrastructure. It supports Dash, Shiny, Grado, Streamlit and Flask frameworks, and apps are automatically deployed and managed in Databricks or in a user’s preferred integrated development environment (IDE).
“You can build, fine tune, train, and serve ML (machine learning) models on top of your data directly inside Databricks,” said Niyogi.
To support security, data never leaves Databricks, he explained, and the app is managed by the company’s data governance tool, Unity Catalog. All users are authenticated through OIDC/ OAuth 2.0 and single sign on (SSO).
“Securing applications can become very difficult,” said Niyogi, as users have to manage controls and add credentials. “It’s often pretty brittle, difficult to manage.”
With Databricks Apps, multiple layers of security, including for physical infrastructure such as VPNs, help to ensure that data doesn’t leave the compliance and regulatory boundary. “You share data when you need to,” said Niyogi. Further, lineage tracking in Unity Catalog allows visibility into what apps and users are accessing what data, and who is making changes.
With this integrated security, some customers have been able to get their first apps into production in days, instead of waiting for weeks for security teams to perform reviews, Niyogi reported.
“Databricks Apps takes advantage of Databricks’ native capabilities to support enterprise-grade data governance and to trace data back to its original source,” said Park.
By choosing a serverless deployment method, Databricks doesn’t constrain app storage or compute. The user experience is also “fairly straightforward,” with standard Python frameworks and templates, he said.
“This experience is not unique, but provides parity with other development environments,” said Park.
He pointed out that Databricks Apps has a lot of competition from business intelligence vendors supporting data apps such as Tableau, Qlik, Sisense and Qrvey. It also vies for market share with “mega vendors” including Microsoft, Oracle, SAP, Salesforce, ServiceNow, and Zoho. Then there are low-code and no-code apps such as Mendix, Appian, and Quickbase at the fringes of the market.
The most important “tactical capabilities” Databricks brings to the table with the new platform, Park noted, is the ability to reuse existing governance, launch from an open-ended serverless environment, and provide a single tool to manage data, infrastructure, and code applications all at once.
“This announcement is consistent with the current brand promise of Databricks as a ‘data intelligence platform’ rather than just being a data insight or data discovery platform,” said Park.
Interfaces that map to data models
Ideal use cases for Databricks Apps include AI applications, analytics, data visualization, and data quality monitoring, said Niyogi.
For instance, a marketing team may create customized dashboards to visualize campaign performance metrics. They could also incorporate AI to perform sentiment analysis on customer feedback, or predictive modeling for forecasts, customer segmentation, or fraud detection.
Databricks customer SAE International, for example, used the platform to turn a retrieval augmented generation (RAG) proof of concept into a branded application that answers questions based on the aerospace company’s knowledge base. And IT services and consulting company E.ON Digital Technology incorporated the platform into its DevSecOps processes to test new features. Other early users include open-source data science company Posit and data apps platform Plotly.
“You can build an interface that really maps to your data model,” said Niyogi.
As Databricks has adopted the platform internally, its own employees have built tools to help improve personal self-management, he said. “Application building is fun,” said Niyogi. “AI development tools unlock people’s creativity. We’re excited to see what our customers build.”
