The Autocomplete That Changed My Workflow

There's a moment in data engineering work where you realize you're typing the same structural boilerplate for the fifth time that week — the same null-handling, the same logging setup, the same exception wrapper you copy from the previous file and adjust. It's not creative work. It's scaffolding. And it consumes more time than it should for someone who already solved this problem three months ago.

That's the gap GitHub Copilot walked into for me, not as a replacement for thinking, but as something that already understood the scaffolding because it had seen enough of it to pattern-match reliably.

Setting Expectations Correctly

I'd been using ChatGPT and GPT-4 as a thinking partner for a while — the rubber duck that talks back, the tool that lets me externalize half-formed ideas and get coherent pushback. That was already valuable. But it required context transfer every single session. Every time I opened a chat, the model knew nothing about my project, my stack, or my conventions. I was re-briefing the same facts repeatedly to get useful output.

Copilot is different in one specific way: it reads your open file, adjacent files, and function signatures. It doesn't know your project history, but it knows what you're building right now. That's a smaller claim than "AI assistant," and it's a more honest one. Copilot is an autocomplete engine with enough context to guess your intent — and when it guesses right, it saves real time.

The expectation problem is when people come in expecting GPT-level reasoning inline. Copilot is not going to reason about your entire architecture. It's going to see that your function is called ingest_from_source, look at your imports, and suggest the shape of the function body based on similar patterns in its training data. Sometimes that's exactly right. Sometimes it's confidently wrong in ways that look correct until you run it.

That calibration took about two weeks to establish. After that, it stopped feeling like a magic trick and started feeling like a well-trained autocomplete that earned its keep.

Where It Immediately Earned Its Keep

The first place I noticed real-time savings was in the repetitive structural parts of data engineering work — the stuff that isn't intellectually interesting but has to be exactly right. Schema definitions. Type conversions. Null-handling patterns. Connection setup for sources I've connected to dozens of times before.

Here's an example of what Copilot gets right without trying:

def read_source_table(
    connection: pyodbc.Connection,
    schema_name: str,
    table_name: str,
    batch_size: int = 10_000,
) -> Iterator[list[dict]]:
    cursor = connection.cursor()
    cursor.execute(f"SELECT * FROM [{schema_name}].[{table_name}]")

    while True:
        rows = cursor.fetchmany(batch_size)
        if not rows:
            break
        columns = [col[0] for col in cursor.description]
        yield [dict(zip(columns, row)) for row in rows]

That's not a complex function. But the first time I typed the signature and Copilot filled in the body correctly — cursor, fetchmany, the column-name extraction from the cursor. description, the yield pattern — I realized this would change how I work. Not because the code was sophisticated. Because I didn't have to look it up or pull it from a previous file. It was just there.

That pattern repeated itself across a week of sessions. SQL generation for standard patterns. Airflow DAG scaffolding. Pytest fixtures for database mocks. The more my codebase looked like code I'd written before (because I'm applying consistent patterns across projects), the better Copilot got at filling in the blanks.

The Context Problem Appears Immediately

Here's the thing nobody tells you up front: Copilot's context window is shallow. It sees your current file and some adjacent files in your editor. It does not understand your project-wide conventions unless those conventions are visible in the files it can read. It does not remember yesterday. It does not know what you named things three directories away unless you have that file open.

This started to surface in week two. I had a naming convention for ingestion modules — a specific prefix pattern, a specific way I structured the class hierarchy — and Copilot kept suggesting the generic pattern instead of mine. Not wrong, just not mine. Every suggestion had to be nudged back toward project convention.

This is a fixable problem, I thought. If context is the issue, I'll provide more context. I started experimenting with what would later become a habit: a Markdown file in the project root that explicitly described my conventions. A simple thing:

# Project Conventions

## Naming
- Ingestion modules: ingest_{source_name}.py
- Main class: {SourceName}Ingestor
- Entry point: IngestorClass.run(config: IngestorConfig)

## Error Handling
- All exceptions logged before re-raise
- Use IngestorError for domain errors, not bare Exception

## Configuration
- IngestorConfig is a dataclass, never a dict
- Secrets come from environment, never from config files

Keeping that file open in the editor shifted Copilot's suggestions noticeably closer to my conventions. Not perfectly — it would still drift — but enough to feel like progress. I filed that away as a promising pattern worth developing further.

The Productivity Shift Is Real, With Caveats

After a few weeks of calibrated use, the time savings were real. Not the "10x faster" marketing number, but meaningful: less time on boilerplate meant more time on the parts of the problem that actually required my judgment. That's the correct framing for this tool. It doesn't make you faster at solving hard problems. It removes friction from the easy parts, freeing up more of your focus for the hard parts.

The caveats are also real. Copilot confidently suggests code that is subtly wrong often enough that you cannot turn off your review instincts. It will generate type annotations that almost match but miss an edge case. It will generate an SQL query that works for 99% of inputs and fails on a specific NULL pattern that matters to your domain. The day you trust a Copilot suggestion without reading it is the day you ship a bug with extra confidence.

That's not a criticism of the tool. It's a calibration note. Used as an accelerator with human review, it's valuable. Used as a replacement for thinking, it's a liability.

What It Changed About How I Work

The shift that stuck wasn't just speed. It was a change in how I approached repetitive patterns. Before Copilot, I had a folder of snippets and templates I'd copy and adapt. After a few weeks with it, I started thinking about patterns differently — less "what template do I paste?" and more "what signal can I give the model so it generates the right shape?"

That's a subtle change in mindset that took a while to name. Writing code became partly about writing code and partly about writing code that made the next completion better. Meaningful function names. Explicit type annotations. Clear docstrings for non-obvious behavior. Things I was already doing from good practice, but now reinforced because they also made the tool more useful.

The markdown convention file was the first experimental step toward something I didn't yet have a name for. The idea that you could shape an AI tool's behavior by curating the context it had access to. That idea was going to get a lot more interesting.

If you've been sitting on Copilot and wondering whether to commit to it, my take is: yes, but calibrate your expectations before you start. It's a force multiplier on pattern-heavy work, not an autonomous developer. If you've been using it a while and found a better way to handle the convention-drift problem, I'd genuinely like to hear it. As always, I'm here to help.