Five Years of "They're Different Tools"
Azure Synapse Analytics launched in 2019 with Synapse Pipelines embedded in it. From the beginning, the Microsoft messaging was: "ADF for enterprise data integration teams, Synapse Pipelines for analytics co-location." Two tools for different use cases. Different teams. Different workloads.
For five years, I've been asking anyone who would answer me: what is the actual technical difference? The answer has gotten vaguer every year.
In March 2023, with the unified platform announcement clearly weeks or months away, it's worth tracing this story from the beginning. Because the pattern it follows is not new — it's how Microsoft data tools have always evolved.
The ADF v2 Origin Story
ADF v1 launched in 2014 as a cloud ETL service. The v1 architecture — slices, availability windows, complex scheduling — was a good-faith attempt at cloud-native batch orchestration that didn't age well. The v2 redesign (2017-2018) was a full rearchitecture: new trigger model, parameterization, integration runtimes, git-deployable JSON. ADF v2 GA'd in 2018.
At v2 GA, ADF was the only Microsoft cloud orchestration product. Clean, clear product line.
The Synapse Launch (2019)
Synapse Analytics launched in 2019 (preview; GA 2021). It was explicitly positioned as the "unified analytics platform" — SQL, Spark, and pipelines in one workspace. Synapse Pipelines was the orchestration layer inside that workspace, using the same underlying engine as ADF. Same connector library. Same pipeline JSON. Same expression language.
Microsoft's differentiation: "ADF if you're building a standalone integration platform; Synapse Pipelines if you're co-locating with Synapse analytics compute."
That's a use-case differentiation, not a technical differentiation. And use-case differentiations erode when the use cases overlap — which they always do in enterprise data platforms.
The Convergence (2019-2023)
Watch what happened to the feature gap over five years:
2019: Synapse Pipelines lacks a few ADF features at launch (some connector types, some IR options). ADF still has a meaningful technical lead.
2020: The connector libraries converge. The activity sets converge. The expression language is identical.
2021: Synapse Pipelines gets git integration, CI/CD via ARM templates, Managed VNet equivalents. The technical gap is now: ADF has Azure-SSIS IR; Synapse Pipelines does not. That's it.
2022: The feature sets are effectively identical except for SSIS-IR. Microsoft's "different tools for different teams" messaging has no technical foundation left to stand on.
2023: A unified platform announcement is coming that will name the product both ADF and Synapse Pipelines were always becoming.
The Pattern: SSIS Did This First
This convergence story is not unique to ADF and Synapse Pipelines. SSIS followed the same arc.
SSIS launched as a standalone SQL Server tool in 2005. It was the on-premises ETL platform for Microsoft shops for fifteen years. As Azure became the target platform, SSIS got the Azure Feature Pack (Azure-specific connectors), then the Azure-SSIS Integration Runtime in ADF (run SSIS packages in managed cloud infrastructure). The pattern: SSIS started standalone, got integrated into SQL Server, then got absorbed into ADF as a compatibility layer.
ADF started standalone, spawned a parallel version in Synapse Pipelines, and is now being absorbed into a unified platform called Fabric (I'm told). The timeline is different — SSIS took twenty years; ADF is doing it in ten. The pattern is the same.
Microsoft data tools consolidate upward into larger platforms. This is not a criticism; it's an observation about how enterprise software evolves. The tool becomes an experience inside a platform, and the platform becomes the product.
What the Five-Year Convergence Cost
Five years of parallel products with nearly identical capabilities meant five years of duplicated engineering effort (at Microsoft), five years of confused product choices (for customers), five years of "which one should we use?" conversations with no clean answer.
The cost is real. Enterprise data teams that chose Synapse Pipelines in 2019 and built their entire integration layer inside Synapse have different operational concerns than teams that built on standalone ADF. Both groups are going to need migration guidance when the unified platform launches. Neither group made a wrong choice — but both groups now face a transition that wouldn't exist if Microsoft had been clearer about the product roadmap in 2019.
The unified platform announcement should include a clear migration path for both groups. If it doesn't, that's the first question to ask at the product roadmap session.
I'll be there. Watch this space.