The Hive Metastore that ships with Databricks has worked fine for years as a catalog. What it has never had is governance: no column-level access control, no cross-workspace data sharing, no lineage tracking, no centralized policy management. Unity Catalog — announced in 2021 and rolling out through 2022 — is Databricks' answer to the governance problem.
The Problem With the Old Hive Metastore
The default Hive Metastore is workspace-scoped. Each Databricks workspace has its own metastore. Tables in workspace A aren't visible in workspace B without mounting the same storage path and re-registering the tables manually. In organizations with multiple workspaces (development, staging, production, or team-specific workspaces), this creates catalog fragmentation — the same table registered in three places with potentially inconsistent definitions.
Access control in the Hive Metastore is cluster-level. If you have access to the cluster and the underlying storage account, you can read any table. There's no column-level or row-level security built in. Sensitive columns (SSN, salary, health data) require either separate tables or application-level masking — neither of which is auditable via the metastore.
What Unity Catalog Changes
Unity Catalog is a separate governance layer that sits above workspaces and manages data access across them. It introduces a three-level namespace:
-- Old namespace (two levels): database.table
SELECT * FROM analytics.customers;
-- Unity Catalog namespace (three levels): catalog.database.table
SELECT * FROM production.analytics.customers;
SELECT * FROM development.analytics.customers; -- same table, different catalog
The catalog is the new top-level container. You can have multiple catalogs per organization — production, development, external_vendor, sandbox — each with their own databases and tables, managed by Unity Catalog centrally.
Cross-Workspace Access
-- In workspace A (production environment)
SELECT * FROM production.analytics.customers; -- works
-- In workspace B (data science sandbox), same query
SELECT * FROM production.analytics.customers; -- also works, with same access controls
Unity Catalog's metastore is account-level, not workspace-level. All workspaces in the same Databricks account that are attached to the same Unity Catalog metastore share the same catalog view. One place to manage table definitions, one place to manage permissions.
Fine-Grained Access Control
-- Grant table access
GRANT SELECT ON TABLE production.analytics.customers TO sales_team;
GRANT MODIFY ON TABLE production.analytics.orders TO etl_service_principal;
-- Column-level permissions
GRANT SELECT ON TABLE production.analytics.customers
TO COLUMN (customer_id, name, region) analyst_group;
-- analyst_group can't see email, phone, or ssn columns
-- Row-level filtering (applied as a function)
CREATE ROW FILTER filter_by_region ON production.analytics.customers
AS (region) -> CURRENT_USER() = CONCAT('analyst-', region);
-- Each analyst only sees their region's customers
Column masking and row filtering are enforced at the catalog layer — not in application code, not in views that can be bypassed. Any query against the table, from any workspace, through any BI tool connected via JDBC/ODBC, goes through Unity Catalog's access control.
Data Lineage
Unity Catalog automatically tracks lineage for SQL queries and DataFrames: which tables were read to produce which tables. In the Databricks UI, you can open any table and see its upstream sources and downstream consumers. This answers "if I change this table's schema, what breaks?" without manually tracing it through pipelines.
What Unity Catalog Doesn't Change
Unity Catalog governs metadata and access. It doesn't move data or change storage. Your Delta tables still live in ADLS or S3. Unity Catalog adds a governance layer on top of the storage you already have.
Migrating from the old Hive Metastore to Unity Catalog requires registering existing tables in the new catalog namespace. The data doesn't move; you're re-pointing Unity Catalog at the same storage paths you already use. Databricks provides migration tooling, but the process requires planning around namespace changes — code that was running SELECT * FROM analytics.customers needs to become SELECT * FROM production.analytics.customers.
For new Databricks deployments starting in 2022, Unity Catalog is the right default. For existing deployments, migration is worth planning — especially for organizations where multiple teams share data and today's "anyone with cluster access can see everything" model will eventually be a compliance problem.