Migrating from ADF to Microsoft Fabric Data Factory: What to Know Before You Start

Not "Should I Migrate?" but "When and How?"

Fabric Data Factory is in preview. GA is November 2023. Clients are already asking me about migration timelines. The short answer: wait for GA, then migrate deliberately. The long answer is this post.

Let me break down what actually moves, what requires manual work, and where the operational model changes enough that you need to redesign — not just relocate.

What Moves Without Significant Work

Pipeline Definitions

The pipeline JSON schema is compatible between standalone ADF and Fabric Data Factory. A pipeline that works in ADF works in Fabric Data Factory. This includes ForEach, Until, If Condition, all the data movement activities, and the expression language throughout. The visual canvas in Fabric Data Factory renders ADF pipelines correctly.

There is no automated migration tool (at preview time). You'll need to recreate pipelines in Fabric. For large factories, this is where having good pipeline documentation (which I recommended you create in January) pays off.

Datasets

Dataset definitions transfer. The dataset schema, format, and linked service binding carry forward. The same caveats on automation: no one-click import at preview time.

Linked Services (Most Types)

Most linked service types — Azure Storage, Azure SQL, Databricks, Synapse, REST, ODBC, SaaS connectors — transfer without issue. The configuration options and authentication methods are the same.

Exceptions: Azure-SSIS IR linked services (SSIS-IR doesn't exist in Fabric), and Self-Hosted IR linked services (setup process differs — test before assuming direct transfer).

Triggers

Schedule triggers, tumbling window triggers, and storage event triggers transfer. Configuration options are the same.

What Requires Manual Work

The Git Integration Model — This Is the Big One

Standalone ADF: git integration is per-factory. The factory connects to a repo. Pipeline authoring happens in a feature branch. Publish generates ARM templates in the adf_publish branch. CI/CD deploys ARM templates to dev/test/prod environments using Azure DevOps pipelines.

Fabric: git integration is per-workspace. The workspace connects to a repo. All Fabric items (pipelines, notebooks, datasets, reports) sync together. No adf_publish branch. No ARM templates. CI/CD uses Fabric's workspace-level deployment APIs.

This is not a reconfiguration — it's a redesign. Your Azure DevOps pipelines that trigger npm publish, deploy ARM templates, run pre/post trigger scripts — none of that applies in Fabric. You need to build new deployment pipelines using Fabric's deployment APIs and the Fabric Git integration model.

The good news: the Fabric deployment model is simpler. No generated ARM templates, no template size limits, no machine-generated JSON to diff in PRs. The workspace sync model is closer to "git push your pipeline definitions and they deploy" than ADF's publish-then-deploy flow.

The work: redesigning your CI/CD pipelines takes time. Plan for it. Don't assume your existing ADO pipelines transfer.

Global Parameters → Workspace Configuration

ADF global parameters (factory-wide values used across pipelines) don't have a direct equivalent in Fabric. The Fabric equivalent is workspace-level configuration variables. The concept is the same; the configuration surface is different. Map your global parameters to Fabric workspace variables before migration, and update all pipeline expressions that reference them.

Integration Runtime Setup

Azure IR: available in Fabric, same concept, minimal setup difference.

Self-Hosted IR: available in Fabric, but the setup UI and registration process differ. Don't assume your existing SHIR nodes register to Fabric the same way they registered to standalone ADF. Test the registration process in a non-production environment first.

Azure-SSIS IR: not available in Fabric. If you have SSIS-IR workloads, they stay on standalone ADF. Plan to maintain a standalone ADF instance for SSIS workloads while migrating everything else to Fabric.

Migration Sequencing Advice

Do not migrate everything at once. The failure mode is: migrate the entire factory to Fabric, discover a gap (SHIR setup difference, CI/CD model change you didn't anticipate, a linked service type that behaves differently), and now you're debugging in production.

Sequence:

  1. Start with a low-criticality, high-complexity pipeline. Something complex enough to test multiple activity types, but not on the critical path if it breaks. Understand what actually happens in Fabric with your specific pipeline patterns.
  2. Build and test the new CI/CD model in parallel with existing ADF deployments. Don't cut over production CI/CD until the new model is validated end-to-end in non-production.
  3. Migrate pipeline by pipeline, in criticality order (low to high). Run parallel — keep ADF production running, migrate pipelines to Fabric, validate in Fabric, then decommission from ADF.
  4. Migrate SSIS-IR workloads last (or never). Keep SSIS-IR workloads on standalone ADF until there's a Fabric path for them, which may be a long time coming.

When to Start

Not during preview. Fabric Data Factory GA is November 2023. Preview products change. Configuration options shift. APIs evolve. Building production migration plans against a preview product is a recipe for rework.

Use the preview period to learn, test, and plan. Build the new CI/CD model in a sandbox. Identify which pipelines have SSIS-IR or complex SHIR dependencies. Document your global parameters. Get familiar with Fabric workspace administration.

Migrate after GA. Deliberately. In sequence. As always, I'm happy to help you build the migration plan.

Read more