Keynote Day 1: Ali Ghodsi and Jensen Huang

Day one keynote. Ali Ghodsi takes the stage, runs through the vision, and then Jensen Huang walks out. The room — and I mean this literally — changed temperature. If you want a barometer for how seriously the enterprise data world is taking GPU-accelerated AI right now, watch a room of sixteen thousand data engineers go slightly reverent when the CEO of NVIDIA shows up to talk about doing AI on Databricks.

Here's what I took away from it, once I got past the spectacle.

The NVIDIA Partnership Is Structural, Not Promotional

The Databricks–NVIDIA integration isn't "our product runs on their GPUs." It's deeper than that. What Databricks is building — training, fine-tuning, serving models at inference — all of it requires GPU infrastructure that most enterprises don't have and can't procure fast enough. NVIDIA's NIM microservices framework, running on Databricks, means you can deploy optimized model inference endpoints without managing the GPU cluster configuration yourself.

For a data engineer who's been spending the last two years building Delta tables and orchestrated pipelines, this matters because it closes the gap between "we have data" and "we have a deployable AI application." The data is in the lakehouse. The model runs on Databricks. The governance is in Unity Catalog. That's the pitch, and for the first time it's actually plausible as an end-to-end story.

GPU Scarcity Is a Real Constraint and Nobody Wants to Say It Out Loud

Here's the thing that went mostly unstated on the keynote stage but was very much discussed in the hallways: GPU availability is still a genuine bottleneck for serious AI workloads. The large training runs, the high-throughput inference endpoints, the fine-tuning jobs on proprietary data — all of these compete for H100s and A100s that are either allocated for months or priced at a level that makes most enterprise ROI models look shaky.

Databricks' answer is to push the efficiency story: fine-tune a smaller model on your data rather than running the large foundation model at inference time. Use retrieval-augmented generation to add context without retraining. Mosaic AI Training is built around the argument that you don't need a 70B parameter model if you have a well-trained 7B model with access to your specific data. That's a defensible architectural position, but it's also partly a response to hardware constraints that aren't going away in 2024.

What Databricks Is Trying to Become

The Data + AI Summit used to be a conference about data. This year it was a conference about AI that happens to run on data infrastructure. The framing shift is deliberate. "Data Intelligence Platform" is the new positioning — not "data lakehouse platform" — and every product announcement maps to either training AI on your data, serving AI from your data, or governing AI with your data.

What Databricks is trying to become is the AWS of enterprise AI: the platform where AI applications get built because the training data, the models, the vector stores, the feature engineering, and the governance all live in one coherent system. Whether they succeed depends on whether enterprises actually want that consolidation or whether the fragmented multi-tool AI stack wins out.

My read: there's a meaningful segment of enterprises who are exhausted by the fragmentation and will pay for consolidation. That's a real market, and Databricks is in the best position I've seen them in to serve it. As always, I'm here to help.