The ADF expression language is the thing that makes parameterized pipelines actually work. It's the DSL for wiring dynamic behavior into your pipeline configuration: reference parameters, pull values from previous activity outputs, compute dates, build dynamic strings. It runs inside the ADF orchestration layer — not in Spark, not in SQL — and understanding its quirks is the difference between pipelines that work and pipelines that fail mysteriously at 3 AM.
The Basics
Expressions start with @ and appear anywhere in your pipeline configuration that accepts a string. If a field shows a data type toggle (static/dynamic), you're in expression territory. Common patterns:
/* Reference a pipeline parameter */
@pipeline().parameters.tableName
/* Reference activity output */
@activity('LookupConfig').output.firstRow.SourceTable
/* String concatenation */
@concat('staging_', pipeline().parameters.tableName)
/* Date formatting */
@formatDateTime(pipeline().TriggerTime, 'yyyy-MM-dd')
/* Current UTC time */
@utcNow()
/* Add days to a date */
@addDays(utcNow(), -1)The @ prefix tells ADF to evaluate this as an expression rather than a literal string. If you want a literal string that starts with @, escape it: @@.
Activity Output References
This is where most of the power lives. When a Lookup Activity, Web Activity, or Copy Activity completes, its output is available to downstream activities.
/* Lookup Activity - single row result */
@activity('GetConfig').output.firstRow.WatermarkColumn
/* Lookup Activity - multiple rows (for ForEach) */
@activity('GetTableList').output.value
/* Copy Activity - rows copied */
@activity('CopySalesData').output.rowsCopied
/* Copy Activity - data read in bytes */
@activity('CopySalesData').output.dataRead
/* Web Activity - JSON response body */
@activity('CallAPI').output.ResponseThe Lookup Activity output.value is an array of objects. You pass this directly to a ForEach Activity's Items field. Each iteration gets one row as the @item() object:
/* Inside a ForEach, reference the current item's fields */
@item().SourceTable
@item().TargetSchema
@item().WatermarkColumnConditional Logic
/* Basic if/else */
@if(
greater(activity('CountRows').output.firstRow.RowCount, 0),
'HasData',
'Empty'
)
/* Check for null */
@if(
empty(pipeline().parameters.optionalParam),
'default_value',
pipeline().parameters.optionalParam
)
/* String equality */
@if(
equals(pipeline().parameters.environment, 'prod'),
'production_container',
'dev_container'
)Comparison functions: greater(), less(), greaterOrEquals(), lessOrEquals(), equals(), not(), and(), or().
String Functions
/* Convert to uppercase/lowercase */
@toUpper(pipeline().parameters.tableName)
@toLower(pipeline().parameters.tableName)
/* Trim whitespace */
@trim(pipeline().parameters.inputString)
/* Replace substring */
@replace(pipeline().parameters.fileName, '.csv', '.parquet')
/* Check if string contains a value */
@contains(pipeline().parameters.env, 'prod')
/* String length */
@length(pipeline().parameters.tableName)
/* Substring */
@substring(pipeline().parameters.tableName, 0, 10)Date and Time
/* Format current UTC time */
@formatDateTime(utcNow(), 'yyyyMMdd')
/* Yesterday's date for incremental loads */
@formatDateTime(addDays(utcNow(), -1), 'yyyy-MM-dd')
/* Tumbling window trigger boundaries */
@formatDateTime(trigger().outputs.windowStartTime, 'yyyy-MM-dd HH:mm:ss')
@formatDateTime(trigger().outputs.windowEndTime, 'yyyy-MM-dd HH:mm:ss')
/* Convert string to datetime */
@parseDateTime(pipeline().parameters.startDate, 'yyyy-MM-dd')Type Conversion
/* Convert to integer */
@int(activity('GetCount').output.firstRow.RecordCount)
/* Convert to string */
@string(pipeline().parameters.batchSize)
/* Convert to boolean */
@bool(pipeline().parameters.includeDeletes)Where It Gets Frustrating
There is no loop construct in the expression language. You iterate with ForEach Activity — the expression language is stateless between evaluations. You can't accumulate a value across iterations using expressions alone.
There are no user-defined functions. If you have a complex computation you use in twelve places, you write it twelve times. The alternative is to push the logic into a stored procedure or a Lookup Activity that computes the value once, then reference the activity output everywhere else.
Debugging complex expressions is manual. The expression editor validates syntax but not semantics. If your expression evaluates to an unexpected value at runtime, the only way to inspect it is to add a Web Activity or a Stored Procedure Activity that logs the expression result, then check the activity output in the Monitor view.
Nested expressions with many function calls get unreadable quickly. This is valid but painful:
@concat(
formatDateTime(addDays(utcNow(), -1), 'yyyy/MM/dd'),
'/',
toUpper(pipeline().parameters.tableName),
'_',
formatDateTime(utcNow(), 'HHmmss'),
'.parquet'
)My practice: when an expression exceeds two function calls, push the logic into a Lookup Activity that runs a SQL query or a stored procedure to compute the value. Keep expressions in the pipeline config simple; let SQL handle the complex computation.
The Right Mental Model
The ADF expression language is not a programming language. It's a dynamic value resolver. It runs once when each activity configuration is evaluated, not during data processing. Think of it like SSIS expressions — not like Python or T-SQL.
Work with that model, not against it. Use expressions for simple dynamic values: table names, date partitions, file paths, activity output references. Use SQL stored procedures or Databricks notebooks for business logic that requires loops, accumulators, or complex conditionals. The right tool for the right job.
Once I stopped trying to use ADF expressions for things they weren't designed for, they became genuinely useful. As always, if you've got a specific expression problem you're wrestling with, I'm here to help.