Microsoft announced in September 2018 that Azure Data Lake Analytics is entering a maintenance-only phase, with no major feature work planned. No new regions. No investment in U-SQL tooling. Just: it works, we're not growing it. If you've been following along, you know I&
I have had this exact conversation at least six times with different clients over the past four years: "Can ADF read from SAP?" "Not natively. You need a custom connector, a third-party ETL tool, or a flat file extract from the SAP team." "..."
Posted in: Microsoft SQL
After I finished evaluating the Storm Events Database from NOAA, I was convinced we needed to look for machine recorded events. When you start poking around the
Posted in: Microsoft SQL
This entry picks up the story behind my first data science project predicting hail damage to farms. In this article we identify data quality issues in our first
Posted in: Microsoft SQL
Early on in my new role I was asked to find out how risky it was to offer hail insurance for a given property. If you haven't worked with insurance before
Machine learning models have a talent for finding patterns in garbage data. Give a model training data with a systematic error and it will dutifully learn to predict based on that error. The model validates fine. It performs great in testing. It ships to production. Then the underlying data error
microsoft sql
In my latest professional change I've moved into a role with a bright and shiny title: Data Scientist. I've heard all the hype around Data Science, Machine
The ADF expression language is the thing that makes parameterized pipelines actually work. It's the DSL for wiring dynamic behavior into your pipeline configuration: reference parameters, pull values from previous activity outputs, compute dates, build dynamic strings. It runs inside the ADF orchestration layer — not in Spark, not