Starting the Year With Eyes Open
Every January I write something about where ADF stands. This January feels different. Not because the product changed dramatically in the last twelve months, but because the organizational signals have gotten loud enough that pretending otherwise would be dishonest.
2023 is almost certainly the last year Azure Data Factory exists as the primary product name in Microsoft's analytics strategy. The consolidation I've been predicting is coming. My bet is May, at Microsoft Build.
Here's what that means for you, your team, and the ADF investment you've already made.
What the Signals Actually Say
The ADF feature roadmap has slowed. That's the clearest signal. In 2019 and 2020, ADF shipped major capability: Data Flows GA, Managed VNet, the v2 parameterization model fully mature. In 2022, the headline ADF announcement was the Managed Airflow integration — real, useful, but not a fundamental expansion of the core platform. Compare that to the Synapse Pipelines roadmap, which is moving faster in 2022.
This is what happens when a product team's attention shifts to a successor. The existing product gets maintenance; new capability goes into what's next.
That's not a death sentence. It's a transition. And transitions in the Microsoft data platform ecosystem are long — they're measured in years, not months.
What You Should Do Right Now
Three things.
Keep Building
If you have ADF workloads in flight, keep building them. The announcement I'm expecting in May is not going to require immediate migration. Microsoft doesn't do immediate migrations. They offer transition periods measured in years. Your pipelines will continue running.
More importantly: the patterns you're building — metadata-driven frameworks, parameterized pipeline templates, CI/CD with ARM templates — are going to transfer to whatever the unified platform becomes. I've been saying this since late 2022 and I'll keep saying it until the announcement proves it: the pipeline JSON is the same, the skills are the same, the architecture is the same.
Don't Panic-Migrate to Synapse Pipelines
I've had three clients in the last quarter ask if they should migrate their ADF workloads to Synapse Pipelines ahead of the product announcement. My answer each time: no.
Here's the logic: if Microsoft announces a unified platform that absorbs both ADF and Synapse Pipelines (which is what I expect), the migration you did from ADF to Synapse Pipelines will have accomplished nothing. You'll have paid the migration cost for zero benefit, and then you'll potentially face another migration to the unified platform anyway.
Wait for the announcement. Understand what the migration path looks like. Then migrate once, to the right destination.
Document Your Factory
Whatever platform is coming, migration requires understanding what you have. If your factory has grown organically over three years — pipelines added by multiple engineers, some documented, some not — now is a good time to do a factory audit. What pipelines exist? What do they do? What are their dependencies? What integration runtimes do they use? What linked services? Which ones use the Azure-SSIS IR (because SSIS-IR may not transfer to the new platform)?
A documented factory migrates faster than an undocumented one. Do the documentation work now, before the pressure of an announced timeline.
What to Watch at Build 2023
Microsoft Build is in May. If my prediction is right, the unified platform announcement happens there. When it does, pay attention to four things:
The underlying storage model. Is there a shared storage layer? How does it relate to ADLS Gen2 and Delta Lake?
The pipeline model. Is it the same JSON schema as ADF? Are existing pipeline definitions compatible?
The SSIS-IR question. Does the Azure-SSIS Integration Runtime exist in the new platform? If not, what's the path for shops running legacy SSIS packages?
The standalone ADF commitment. Does Microsoft commit to continuing standalone ADF as a supported service? For how long?
The answers to those four questions will tell you everything you need to know about what to do next.
I'll be watching. I'll be writing. As always, I'm here to help you navigate whatever comes.