Every few years a tool shows up that makes you stop and reconsider your defaults. For most of us doing ETL on the Microsoft stack, SSIS is that default. It works, it's mature, you know exactly where to look when it breaks. Then Azure Data Factory hit preview
Every data project I have ever been on started with someone telling me what they needed the system to do. Not how — what. "We need to load customer orders from the source system into the warehouse by 6 a.m. every day." "We need to flag records
Nobody tested their stored procedures in 2010. I mean that almost literally — the pattern was: write the procedure, run it manually with a few sample inputs, see if it does approximately what you wanted. Automated tests for database logic were theoretically possible but practically nonexistent in most shops. SSDT and
There is a version of this story in every client engagement I have been on. The first pipeline works, so someone immediately wants to add retry logic, parameterization, alerting, data quality checks, and a metadata-driven configuration framework — all at once, in the same sprint. The result is always the same.
A dacpac contains the schema model. A publish profile controls how that model gets applied to a specific environment. Without publish profiles, every deployment would use the same settings — the same permissiveness for destructive changes, the same connection details, the same behavior for out-of-sync objects. With publish profiles, dev, staging,
microsoft sql
Welcome back to my mini-series on Powershell, SQLPSX, and the awesome things you can do with your packages through POSH. You can catch up by reading this, this
microsoft sql
You can follow the discussion on PowerShell Management for Your SSIS Packages without having to leave a comment. Cool, huh? Just enter your email address in the
microsoft sql
Job ApplicationThanks for checking out this job, if you'd like to get more information, please fill out the following information, and we can get to work right away! * indicates required field Name: * Email: * Message: * Resume Acceptable file types: doc,pdf,txt,gif,jpg,jpeg,png.Maximum file size: 1
A client called me in to "fix a broken SSIS package." One package. Singular. I opened it and found something that had been growing for three years: extraction, validation, transformation, error logging, archival, and a handful of hard-coded business rules buried in a Script Task someone had labeled
By 2014, the two camps in database deployment tooling were well-defined. In one corner: migration-based tools — Flyway, Liquibase, DbUp — where you write numbered scripts representing each change and the tool tracks which ones have been applied. In the other: desired-state tools — SSDT dacpac model, ReadyRoll — where you describe the target
microsoft sql
Last time, I introduced you to SQLPSX. Then we spun through our SSIS packages looking for a specific variable. That's where I started my PowerShell managing
SSIS packages are not bad. I want to be clear about that before what follows, because it might sound like a condemnation. SSIS is a mature, capable ETL tool that handles complex data transformations, handles errors gracefully, and integrates well with the SQL Server ecosystem. The problem isn't