Generative AI front and center at Databricks’ Data + AI summit, following MosaicML acquisition

Databricks turned the spotlight on generative AI at its annual Data + AI summit, as it announced a host of new Lakehouse AI innovations.

This focus on generative AI, the company said, “highlights the inflection point reached with the rise in the popularity of large language models (LLMs)”.

In April, the company launched what it calls the first truly open-instruction tuned LLM, Dolly 2.0, that powers apps such as text summarizers and chatbots and allows commercial use by independent companies and developers.

It recently also spent $1.3 billion to acquire an AI startup, MosaicML to enable businesses to build generative AI models with their own data.

Lakehouse AI seeks to offer the same “data-centric approach to AI”, the company said, by unifying the data and AI platform so customers can develop their generative AI solutions faster and more successfully by using foundational SaaS models to train their own custom models with their enterprise data.

Newly announced capabilities include:

1. Vector Search – Enables developers to improve the accuracy of their generative AI responses through embeddings search. Embeddings are numerical representations of text that capture its semantic information, making it easier for computers to understand relationships between concepts. It also automatically creates and manages vector embeddings from files in Unity Catalog, Databricks’ flagship solution for unified search and governance. Via integrations with Databricks Model Serving, developers can improve the response from models by adding query filters to the search.
2.  Fine-tuning in AutoML – Brings a low-code approach to allow customers to fine-tune LLMs using their own data, which results in a model produced by AutoML without having to send data to a third party. Integrations with MLflow, Unity Catalog and Model Serving also enables the sharing of the model within an organization.
3. Curated open source models – The Databricks Marketplace offers a curated list of open source models, including models for various generative AI use cases such as instruction-following, summarization, and image generation.

Further, the company announced MLflow 2.5, the latest version of the Linux Foundation open source project MLflow. Updates to MLflow 2.5, slated to go live in July, include:

1. MLflow AI Gateway – allows centralized management of credentials for SaaS models or model APIs and provides access-controlled routes for querying, enabling integrated workflows. Developers can also swap out the backend model to improve cost and quality, as well as switch across LLM providers. It also enables prediction caching to track repeated prompts, and rate limiting to manage costs.
2. MLflow Prompt Tools – No-code visual tools allowing users to compare models’ output, based on a set of prompts, which are automatically tracked within MLflow.

Other announcements made at the summit include:

1. Update to Databricks Model Serving to enable GPU-based inference support for LLMs, with up to 10x lower latency time and reduced costs.
2. Introduction of Databricks Lakehouse Monitoring to better monitor and manage all data and AI assets within the Lakehouse.
3. Lakehouse Federation capabilities, which allow customers to discover, query, and govern data across all of their data platforms from within Databricks without moving or copying the data first, hence eliminating data silos.
4. Launch of Delta Lake 3.0, introducing Universal Format (UniForm), which allows data stored in Delta to be read from as if it were Apache Iceberg or Apache Hudi.
5. Launch of LakehouseIQ, which uses generative AI to understand jargon, data usage patterns, organizational structure, and more, to answer questions within the context of a business.