ADF Connector Update 2018: SAP Finally Shows Up

Shannon Lowder

Shannon Lowder

— 3 min read

I have had this exact conversation at least six times with different clients over the past four years:

"Can ADF read from SAP?"

"Not natively. You need a custom connector, a third-party ETL tool, or a flat file extract from the SAP team."

"..."

2018 is the year that conversation changes. SAP connectivity is now native in ADF. Let me walk through what shipped and what the 2018 connector story looks like overall.

SAP Connectors

SAP Table Connector

Read directly from SAP transparent tables and cluster tables via RFC. This covers the majority of operational data extraction scenarios — reading from SAP ECC or S/4HANA tables like BSEG, EKPO, MARA, and so on.

The setup requires the SAP NCo (SAP .NET Connector) library installed on the Self-Hosted Integration Runtime machine. SAP NCo is a separately licensed SAP component — you'll need your SAP Basis team involved. Once the NCo library is on the SHIR server, the ADF linked service configuration is straightforward: application server host, system number, client, logon credentials.

/* Linked Service configuration (key properties) */
{
  "type": "SapTable",
  "typeProperties": {
    "server": "your-sap-host",
    "systemNumber": "00",
    "clientId": "100",
    "userName": "ADF_EXTRACT_USER",
    "password": { "type": "AzureKeyVaultSecret", ... },
    "logonGroup": "PUBLIC"
  }
}

Performance note: large SAP table extracts benefit from partitioning. The connector supports partitioning by row ranges, which parallelizes the extraction across multiple RFC calls. For tables with tens of millions of rows, configure parallelism to match your SHIR node count.

SAP BW Open Hub Connector

For clients running SAP BW (Business Warehouse), the Open Hub connector reads from BW Open Hub Destinations. This is the standard SAP BW data extraction pattern — BW processes data internally, writes to an Open Hub Destination, ADF reads from there. If your client has existing Open Hub Destinations defined, this connector plugs directly into that pattern.

SAP HANA Connector

Direct connectivity to SAP HANA databases using the HANA JDBC driver. Also requires the ODBC driver on the SHIR machine. Reads from HANA views, calculation views, and tables. Useful for clients running S/4HANA where you need to access the embedded HANA directly rather than going through RFC.

Snowflake

This one matters. Snowflake has been growing fast as a cloud data warehouse target, and ADF now has native Snowflake connector support.

The connector uses the Snowflake JDBC driver. For reads: standard SELECT queries or table scans. For writes: staged loads via Azure Blob (ADF writes files to Blob, Snowflake bulk-loads from there via a COPY INTO command). The staged load approach is significantly faster than row-by-row JDBC inserts for large datasets.

If you're seeing Snowflake as a target in your client environments — and you will — this connector handles the integration cleanly. The staging approach means you'll need an Azure Blob linked service alongside your Snowflake linked service.

REST Connector Improvements

The REST connector shipped in 2017 but has been substantially improved in 2018. Key additions:

  • Pagination support — configure ADF to follow pagination links automatically. Specify the pagination rule (next page URL in response body, page number parameter, skip/top parameters) and ADF handles the loop.
  • OAuth 2.0 improvements — better support for client credentials flow and authorization code flow. Complex OAuth scenarios that required custom Azure Functions workarounds in 2017 often work natively now.
  • Request body templating — POST requests with dynamic JSON bodies using ADF expressions.

For API integrations that involve paginated endpoints, this is a significant quality-of-life improvement. I've replaced several custom Azure Function wrappers with native REST connector configurations this year.

File Format Connectors

ADF now supports Parquet, ORC, and Avro as native dataset types — not just CSV and JSON. This is important for data lake workloads where your canonical storage format is columnar.

The binary file connector allows raw file copy without any schema parsing — useful when you're moving files whose schema you don't control, or when you need to archive source files exactly as received.

Dynamics 365 and Salesforce Service Cloud

Enterprise CRM connectivity has been a gap for ADF since the beginning. Both Dynamics 365 and Salesforce now have native connectors with reasonable feature coverage:

  • Standard and custom entity reads via the respective APIs
  • Write support for both (create, update, upsert)
  • Dynamics connector uses OAuth with Azure AD app registration
  • Salesforce connector supports bulk API for large volume operations

The Overall Connector Picture

ADF's connector library has crossed a threshold this year. In 2015 and 2016, I routinely had to explain to clients why their source system wasn't supported and discuss workarounds. In 2018, the list of systems that require a workaround is shrinking toward the genuinely obscure.

Enterprise sources that were historically problematic — SAP, Dynamics, Salesforce — are now native. Modern cloud targets — Snowflake, ADLS Gen2, Cosmos DB — are well-supported. The REST connector handles the long tail of API-based sources.

The remaining gaps are increasingly niche: some legacy mainframe formats, specific SAP modules that aren't covered by the three SAP connectors, and some on-premises databases that require custom ODBC drivers that aren't pre-loaded on the SHIR. Edge cases, not the mainstream.

If you're planning a new ADF project and you've been blocked by connector gaps in the past — check the list again. The answer may have changed. As always, I'm here to help if you're evaluating whether a specific source system is covered.

Read more