TabNine and the First AI Completion That Actually Helped

Shannon Lowder

Shannon Lowder

— 2 min read

I've been burned by AI code completion enough times to approach every new entrant in this space with genuine skepticism. IntelliSense completes your method names. Kite autocompletes Python idioms based on what's in scope. These are useful but they're not surprising — they're fast lookups against what you've already written.

TabNine is different, and I want to be precise about how.

The model underneath it is GPT-2, the same architecture OpenAI released earlier this year. TabNine fine-tunes it on code across dozens of languages and runs it locally or via their cloud backend. The result isn't lookup-based completion — it's generative. It predicts what you're about to write based on the full context of the file, not just the current line.

What Surprised Me

The first time TabNine earned my attention was on a Spark job I was writing for a client's event ingestion pipeline. I had written two transformation stages — normalize the schema, then filter out internal traffic. I started typing the third stage, got through the function signature, and TabNine completed the body: the correct column selection, the right filter logic, even the right output path pattern following the convention I'd established in the previous two stages.

It didn't know my data. It knew my pattern. That's the distinction that matters.

For data engineers writing structurally repetitive code — Spark DataFrame transformations, Airflow operator configurations, PySpark schema definitions — that pattern recognition is where the time savings actually live. Not in typing speed; in the cognitive load of remembering the exact API surface for the seventh time.

from pyspark.sql import SparkSession
from pyspark.sql import functions as F

spark = SparkSession.builder.appName("event-ingest").getOrCreate()

# Stage 1 — TabNine saw this...
raw = spark.read.json("s3://data-lake/raw/events/")
normalized = raw.select(
    F.col("userId").alias("user_id"),
    F.col("pageUrl").alias("page_url"),
    F.col("eventTs").alias("event_ts")
).filter(F.col("event_ts").isNotNull())

# Stage 2 — ...and this...
filtered = normalized.filter(
    ~F.col("user_id").startswith("internal_")
)

# Stage 3 — ...and completed this without me finishing the thought:
partitioned = filtered.withColumn(
    "event_date",
    F.to_date(F.from_unixtime(F.col("event_ts") / 1000))
)

Where It Fails

TabNine fails on anything requiring domain knowledge it doesn't have. It doesn't know that your event_ts is in milliseconds, not seconds — it infers that from context if you've been consistent, but the first time you write it, you're on your own. It doesn't know your business rules. It doesn't know that a NULL user_id means "anonymous" in one table and "bad data" in another.

It also generates confident nonsense when there's no pattern to match. If you're writing something structurally novel, the suggestions are often wrong in ways that look plausible — the most dangerous kind of wrong.

The Right Mental Model

TabNine is not a code-writing tool. It's a pattern accelerator. It reduces the friction of expressing a pattern you already know, in a codebase where you've already established conventions. On a new project or a novel algorithm, it's background noise you learn to ignore. On a mature pipeline codebase with consistent naming and structure, it's a meaningful time saver.

The signal that it's actually working: you stop noticing it. The completions become invisible because they're correct often enough that the rejection of a bad suggestion takes no more mental energy than deleting a typo. That's the bar. TabNine clears it for specific data engineering work patterns.

If you've been running it on Python data work and have a different read on where it earns its keep, I'd like to hear it. As always, I'm here to help.

Read more