Lakehouse Interoperability: The UniForm Moment

Shannon Lowder

Shannon Lowder

— 2 min read

Every conference has one announcement that the press covers extensively and one that the practitioners quietly recognize as more significant than the coverage suggests. At DAIS 2024, UniForm was the second kind.

The headline version: Delta Lake tables can now be read natively by Iceberg-compatible engines without copying or converting the data. Snowflake can query your Delta table. BigQuery can query your Delta table. Starburst can query your Delta table. The same underlying Parquet files, the same transaction log, accessible by engines that previously required you to export data to their format first.

This is a bigger deal than it sounds.

Why Interoperability Matters More Than Format Choice

The table format debate — Delta vs. Iceberg vs. Hudi — has been running for several years and has generated a lot of heat. Most of that heat was misdirected. The real question was never which format is technically superior; it was which format your ecosystem would support. If your primary analytics engine is Databricks, Delta is the right choice. If your primary engine is a Trino-based system, Iceberg might make more sense. The "wrong" answer was being locked into a format that forced you to copy data when you needed to use a different engine.

UniForm changes the calculus. If you can store data in Delta and read it natively with Iceberg or Hudi APIs, the format choice becomes much less consequential for downstream flexibility. You pick the format that your primary engine handles best, and you rely on the interoperability layer for everything else.

What Changed for Multi-Engine Architectures

Before UniForm, a common pattern in larger enterprises looked like this: data engineers write to Delta in Databricks, then run a nightly export job to Iceberg or Parquet for consumption by other teams using Athena, BigQuery, or Snowflake. That export job is a maintenance burden, introduces latency, creates a synchronization problem, and doubles your storage costs.

With UniForm enabled on a Delta table, those export jobs go away. The downstream consumers read directly from the Delta table using Iceberg-compatible APIs. No copy, no synchronization lag, no duplicate storage bill.

-- Enable UniForm on a Delta table
ALTER TABLE prod_analytics.gold.daily_order_summary
SET TBLPROPERTIES (
  'delta.universalFormat.enabledFormats' = 'iceberg'
);

-- Iceberg metadata is now maintained automatically alongside Delta metadata
-- An Iceberg-compatible engine reads this table at the same path
-- No exports, no conversions, no separate catalog registration

My Prediction: The Next Five Years Are Won on Interoperability

The data platform wars of the last decade were won on features and performance. The next decade will be different. Every major platform has caught up on features. Performance is good enough almost everywhere that it's rarely the deciding factor in platform selection. What will differentiate winning architectures is whether they can participate in an ecosystem without forcing every adjacent system to adopt their format.

The vendors that commit hardest to genuine interoperability — not "you can export to other formats" but "our format is natively readable by your other tools" — are the ones that will be trusted with the core of enterprise data estates. UniForm is Databricks making that commitment concrete. I expect Snowflake, Google, and AWS to respond in kind. The format war is ending; the ecosystem war is just beginning. As always, I'm here to help.

Read more