Mosaic AI: The Real Foundation for Enterprise GenAI (2024 Edition)

Two months out from DAIS, I've had time to actually use Mosaic AI rather than just watch the demos. The unified branding that Databricks put on this product family at Summit — training, serving, vector search, agent framework, tools catalog — was clearly designed to compete with the fragmented GenAI stacks most enterprises have accidentally assembled. After living with it, I think the unified story holds up better than most vendor "platform" pitches do.

What Mosaic AI Actually Enables Today

The practical workflow that Mosaic AI makes coherent: fine-tune a foundation model on your data in Mosaic AI Training, register it in the UC-backed model registry, deploy it to a Mosaic AI Model Serving endpoint, and govern access through Unity Catalog. Vector search lives in the same catalog namespace as your Delta tables. The MLflow tracking that captures training runs is the same MLflow that tracks serving metrics.

That coherence is the real value proposition. Before this, a typical enterprise GenAI stack looked like: data in Databricks, model in SageMaker, vector store in Pinecone, serving in a custom FastAPI container, lineage tracked nowhere. Every component from a different vendor, nothing governed centrally, audit trail nonexistent. Mosaic AI doesn't replace every component — but it replaces enough of them that you can answer "who has access to this model and what data trained it" without pulling logs from four different systems.

Where It Feels Mature

Model Serving is genuinely production-ready. I've covered it before: traffic splitting, scale-to-zero, traffic governance through UC. This part works. Foundation Model APIs — the ability to call externally hosted models through the same endpoint interface as your fine-tuned models — is a clean abstraction. Your application code doesn't need to know whether it's hitting a Databricks-hosted model or an external API; the serving layer handles it.

Vector Search GA is also solid. The integration with Unity Catalog for access control is exactly right — you don't want a vector index that bypasses the row-level security on the underlying table, and UC-native vector search respects the same permission model as the source table.

Where It Still Feels Early

The Agent Framework is in preview and it shows. The abstraction for composing multi-step agent workflows is there in skeleton form, but the tooling for testing, debugging, and monitoring compound agent executions is immature. Building a production agent on the current framework requires significant custom instrumentation that shouldn't be necessary if the framework is serious.

The Tools Catalog — a registry for the tools and functions that agents can call — is conceptually right but currently thin. If you've seen what a well-designed function-calling registry looks like in other agent frameworks, the current Mosaic AI implementation is early-stage by comparison. Expect this to evolve significantly over the next six months.

What I'd Like to See at Summit 2025

A complete observability story for agentic workloads — not just "the agent ran," but "the agent called these tools, received these results, made these decisions, and here's the trace." That telemetry is table stakes for production agents. A mature testing framework for compound AI systems would also be transformative. Right now, testing an agent pipeline requires too much custom scaffolding. As always, I'm here to help.