I run a real Kubernetes cluster on a stack of arm64 single-board computers. Not a toy — it hosts my Git server, this blog, and the automation I lean on every day. It's cheap, quiet, and sips power, and I'd build it the same way again. But running production workloads on cheap ARM boards has bitten me in ways a cloud VM never would, and every one of those bites was avoidable once I knew where the teeth were. So here's the pitfall catalogue. Read it and skip my scars.

The "arm64" image that isn't

Your whole fleet is arm64, so you pull an image, the manifest proudly lists linux/arm64, the pod schedules, and then it crashes with exec format error. What happened: the image advertises an arm64 platform in its manifest, but the binary inside was compiled for amd64. The multi-arch tag was aspirational. I burned an afternoon on one popular self-hosted app before I accepted the image simply couldn't run on my hardware, and I ripped it out for an alternative that ships genuine ARM builds.

The lesson: don't trust the manifest, test the binary. Before you commit to a piece of self-hosted software, pull its image and actually start it on an ARM node. If it exec format errors, you've saved yourself a migration later. Multi-arch support is a claim until your own hardware proves it.

reclaimPolicy: Delete is a data shredder

This is the one that cost me real data, so let me be blunt about it. A PersistentVolume's reclaimPolicy decides what happens to the underlying data directory when its claim goes away. The default on a lot of provisioners is Delete, and Delete means exactly that: remove the PVC — on purpose, by accident, during a "clean up this namespace" sweep — and the provisioner erases the data on disk with it. No trash, no undo.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: git-data
spec:
  persistentVolumeReclaimPolicy: Retain   # not Delete. ever, for stateful data.

Set Retain on anything that holds state you can't regenerate, full stop. With Retain, deleting the claim orphans the PV instead of shredding it — you clean it up by hand when you actually mean to. I learned this by deleting a claim I thought was disposable and watching my Git server's data directory evaporate. I had a backup. You should assume you won't be so lucky.

A node dies, and its data dies with it

On a real cloud, storage floats free of any one machine. On an SBC cluster, the cheap, fast option is local-path storage — a directory on that specific board's disk — and local volumes are node-pinned. The pod can only run where its data lives. That's fine right up until the board dies, and SBCs die more than servers do: they run their root filesystem off SD cards, and SD cards wear out and corrupt without warning.

This is the one that finally got me to act. After more than a year of trusting my nodes' root storage to SD cards — living with exactly the fragility I'm describing — I'm moving them to NVMe, and naturally I'm doing it right as NVMe prices spike. Timing is a skill I have clearly not mastered. I went with NVMe specifically because the PoE HATs I already run on each node support an NVMe drive too, so I could add real storage without adding another power feed — the whole draw stays minimal, which matters when your "data center" is a shelf of tiny boards. The longer game is bigger: once I can afford to replace my UNAS Pro with something that speaks block storage and iSCSI, the data stops being welded to any one node. That's the actual cure for node-failure sensitivity — when storage is served over the network as blocks, a dead board becomes capacity you reschedule around instead of data you scramble to restore.

When a node goes, everything pinned to its local storage goes with it — not corrupted, just gone, sitting on a disk you can't reach. Two rules keep you out of this hole. First, treat every board as disposable and back its stateful volumes up to somewhere off the node, on a schedule. Second, know that local-path lies to you in smaller ways too: ask it to expand a volume from 5 to 8 gigs and it'll happily accept the change and do nothing. The number updates; the disk doesn't.

Your backup is lying to you

Here's a mistake hiding inside the fix for the last one. I had a migration helper that tarred a volume, checked the archive existed, and then deleted the source. It ran, it "succeeded," and it destroyed the data — because it had tarred a path that didn't exist on the node it was pointed at, producing an empty gzip. An empty gzip is about 20 bytes, and 20 bytes passes a "file is non-empty" test just fine.

# WRONG — an empty archive is ~20 bytes and sails through this
[ -s backup.tar.gz ] && rm -rf "$source"

# RIGHT — validate by entry count, not file size
[ "$(tar tzf backup.tar.gz | wc -l)" -ge "$EXPECTED" ] && rm -rf "$source"

Validate a backup by what's in it — a row count, an entry count, a checksum of a known file — never by whether a file showed up. And if you're migrating node-pinned storage, derive the source location from the volume's own node affinity; never hardcode which board it's on, because the day you hardcode the wrong one is the day your backup is 20 bytes of nothing and your "safe to delete" step isn't.

A kernel upgrade can silently cut the network

I took one node to a new distro release and its networking died in the most confusing way possible: the link showed up, ARP resolved, everything looked connected — and not a single IP packet moved under sustained load. A kernel update had regressed the board's network driver. The tell was that it wasn't a refusal, it was a hang: ports open instantly, transfers stall forever.

I proved it the boring, reliable way — an A/B test. Upgraded nodes wedged; nodes on the old kernel stayed rock solid, two for two versus three for three. Then I rolled the upgraded boards back and stopped. The discipline here is simple and it's the same discipline you'd want anywhere: never march your whole fleet onto a new kernel at once. Upgrade one board, beat on it under real load for a day, and only then let the change spread. On homogeneous cheap hardware, a bad kernel doesn't break one node — it's queued up to break all of them.

The bonus round: when a dead node takes your alerting with it

This is the one that still stings. I own a t-shirt that just says "It was DNS." I wear it more than I'd like to admit, because it almost always is — and this time was no exception. A node died, and because my cluster DNS was running as a single replica, its death browned out name resolution across the whole cluster. Detection worked perfectly — the "node not ready" alert fired right on time. Delivery is where it fell apart: my alert-router couldn't resolve the address of the chat webhook it was supposed to notify, so the alarm never left the building. The monitoring system became collateral damage of the exact incident it existed to report.

Run your cluster DNS with at least two replicas and anti-affinity so no single board can take it down. And put one check outside the cluster entirely — a dead-man's-switch heartbeat that screams if it stops hearing from you. In-cluster monitoring can only tell you about failures that leave the monitoring path intact. The failures that matter most are the ones that don't.

The pattern under all of it

Every one of these comes back to the same mindset shift. A cloud VM lets you pretend the machine is permanent and the storage is safe. A stack of ARM boards won't let you pretend: treat every node as disposable, every local volume as ephemeral until it's backed up off the box, every backup as guilty until a row count proves it innocent, and every fleet-wide change as a thing you test on one board first. Adopt that posture and the platform is genuinely great — cheap, fast, and yours. Fight it, and it will teach you these lessons the expensive way, like it taught me.

If you're running your own boards and you've hit a pitfall I missed — or found a cleaner way to dodge one of these — I'd genuinely love to hear it. As always, I'm here to help!

I've rebuilt the way my agents talk to language models three times in about three months. Each rewrite solved the previous one's biggest pain and then exposed the next one. The version I run today routes everything through a multi-provider gateway called OmniRoute — but that choice only makes sense if you know the two experiments that came before it, because the gateway isn't the interesting part. The shape is. Let me walk you through the lineage, because you're probably about to build some slice of this yourself, and I can save you a rewrite or two.

One thing up front so I don't mislead you: OmniRoute is not something I wrote. It's an off-the-shelf gateway I adopted at the end of this story. What I built — and rebuilt — was everything around it. That distinction turns out to be the whole lesson.

Stage one: the relay that could reach anything and decide nothing

The first version had one goal: make many providers look like one. I wanted a single endpoint my code could call and stop caring which vendor was behind it. The specific itch was unlocking a coding-assistant subscription I already paid for as a backend provider, which meant a device-flow OAuth handshake and a token sidecar to keep it alive.

So I built a relay. A thin service with a provider-adapter contract — a uniform interface, so a new provider was just another adapter you plugged in. It worked. I could hit a dozen providers through one door.

And then I stared at it and realized it was only half of a system. The relay knew how to reach every provider. It had no opinion whatsoever about which one to use, or when, or why. Every routing decision still lived in my application code, hard-coded and duplicated. A relay is transport. Transport doesn't make choices. That's not a bug you fix in the relay — it's a missing layer.

Stage two: the router that made the choices

So I built the missing layer: a reasoning router that sat above the relay and decided things. Which local model handles this? When do I escalate from a small local model to a bigger one? When is it worth reaching for an external paid model, and which one? Those are policy questions — cost, latency, quality — and the router answered them on every request, instrumented so I could see what it chose and what it cost.

The critical design call, the one that survived everything after it: the router did not replace the relay. It sat on top. I now had two clean layers with a clear seam between them — a policy layer that answered "which model and why," and a transport layer that answered "how to reach it." Keep policy out of transport. Write that on a sticky note. It's the load-bearing idea in this entire post, and the reason the next rewrite didn't hurt.

Think of it like a dispatcher and a road network. The dispatcher decides which truck goes where based on cost and deadline; the roads just get you there. You don't teach the roads to make dispatch decisions, and you don't make the dispatcher repave anything.

Stage three: stop hand-rolling the roads

Here's where I made the pragmatic call. My hand-rolled transport layer was fine at a dozen providers. It was not going to be fine at hundreds — plus semantic caching, quota tracking across every vendor, prompt compression, and server-side fallback. I could spend the next year rebuilding a mature gateway, or I could adopt one and put my energy where it actually differentiated me: the policy above it.

I adopted OmniRoute — a gateway that already speaks to hundreds of providers with a stack of routing strategies — and moved my rotation logic down into its combos. A combo is a weighted group of models the gateway rotates through, with fallback built in. The mental shift that mattered: the combo, not the individual model, became my execution unit. My orchestrator stopped iterating over models one at a time. It hands work to a combo and trusts the gateway to try every member. "Each node tries every model" became a guarantee I got for free instead of code I maintained.

That unlocked the resilience patterns I'd been faking:

• Round-robin dispatch to spread load, so I don't concentrate every request on one provider and either burn its paid credits or earn a rate-ban.
• Two kinds of cooldown — a short one for a rate-limited model (back off for a minute) and a long one for a model that's blown its daily quota (park it for the day). A 429 shouldn't crash a run; it should rotate.
• Free-before-paid interleaving — try the free attempts first, and let paid slots fire only after the free ones miss. Cost discipline as a routing rule, not a hope.

The availability floor, and why I stopped trusting my own docs

The most important rung in any combo is the last one: a free, self-hosted local model that's always reachable. I call it the availability floor. When every fancy provider is rate-limited, quota-exhausted, or down, the floor catches the request so the loop keeps moving. No floor means a bad afternoon of "circuit breaker open" errors and dead runs.

And this is where I earned a scar I want you to skip. I had the floor documented. The config file clearly showed a local model as the terminal rung. I'd written "done" and moved on. Weeks later, when runs kept dying on circuit-breaker-open, I finally checked the live gateway — and the running combos had no local member at all, and were set to plain round-robin instead of ordered priority. Round-robin doesn't guarantee the floor gets tried; only an ordered combo does. The doc was right. The system was wrong. They'd drifted, and I'd trusted the doc.

The lesson, which I've now relearned enough times to tattoo on: a documented fix is not a fix. Verify the change is in the file and applied to the live system before you write "done." Especially for routing, where the failure is silent until everything's on fire at once.

Treat the gateway as a dependency you don't control

Adopting someone else's gateway bought me a year of engineering and a new problem: it's a black box I don't govern. It ships breaking changes on its schedule. Its reported pricing isn't always trustworthy — a model that claims to cost nothing might just have a missing price. And it doesn't take my pull requests, so I can't fix any of that upstream.

My answer was to wrap it behind a single anti-corruption seam. Everything my agents see is a trusted projection of the gateway: a curated model catalog I control, and a pricing overlay that refuses to believe a "$0" unless I've explicitly whitelisted it — unknown-price models get excluded, not defaulted to cheap. Upstream churn hits that one seam instead of rippling into every node. And because the seam exists, I could rip out OmniRoute and drop in my own router later without touching a single agent. That's the same instinct as the provider-adapter contract from stage one, just aimed at a bigger dependency: never let an outside thing's shape become your shape.

The throughline

Three rewrites, one pattern: a reasoning router that owns policy, sitting above an aggregation gateway that owns transport, with a hard seam between them. The relay taught me transport isn't enough. The router taught me policy belongs in its own layer. The gateway taught me to buy transport instead of building it — and to fence it off like the untrusted dependency it is. The standalone projects are all retired now. The pattern is the only thing I kept, and it's the only thing that mattered.

If you're wiring up model routing for your own agents, start with the seam, not the gateway. Decide where "which model and why" lives before you pick "how to reach it," and you'll survive swapping the second one out. If you've drawn the line somewhere different — or found a gateway you trust more — I'd love to hear how it's holding up. As always, I'm here to help!

Six months ago I got tired of introducing myself to my own AI assistant. Every session started the same way: re-explain the architecture, re-list the repos, re-state the decisions I'd already made and the mistakes I'd already learned from. The model was brilliant and amnesiac, and I was its full-time memory. So I built it a memory it could read and write — a plain folder of markdown files that both of us edit. Not a vector database, not a SaaS "memory layer." Files. Here's what I've learned about making that work when a human and a fleet of agents are all writing to the same brain, because the writing part is where it gets interesting.

Why files, and why both of us write to them

Start with the constraint that makes this hard: the assistant's context resets every session, and mine doesn't. If the memory only I can edit, it's documentation, and documentation rots because updating it is a separate chore. If it's a memory only the assistant can edit, I can't correct it when it's wrong, and I can't read it when I'm the one who needs the reminder. A shared brain has to be first-class for both of us or it decays.

Plain markdown files win here for an unglamorous reason: everything already speaks them. I can grep them, diff them, edit them in any editor, and put them under version control. The assistant can read them, append to them, and link them together with no special tooling. The format is boring, and boring is exactly what you want for something that has to survive years and a dozen tool changes.

The two hooks that kill cold-start

A pile of notes isn't a memory until the right ones show up at the right time. Two mechanisms do that work, and you can build both.

The first is a small index note that loads at the start of every session — the map of what's stored where. One line per major note: a title, a link, and a one-sentence hook. The assistant reads that first and knows the shape of everything it could pull, without pulling all of it. That single file is the difference between "cold model" and "model that knows which drawer to open."

The second is a retrieval step that fires on each message and surfaces the handful of notes relevant to what I just asked. Ask about scheduling and the scheduling gotchas appear; ask about a client's data and the governance decisions appear. The assistant consults them before it asks me a question I've already answered in writing. The rule I enforce: if the answer might already be in the brain, check the brain before asking the human. That one rule reclaimed more of my time than any prompt-engineering trick.

One fact per file, and a little frontmatter

Keep the atoms small. Each memory is one fact or one tightly-scoped note, not a sprawling document, because small notes are easier to retrieve, dedupe, and retire. Give each one a scrap of frontmatter — a name, a one-line description that's used to judge relevance during recall, and a type so you can tell a durable fact from a personal preference from a pointer to an external resource. That description field earns its keep: it's what a retrieval step reads to decide whether this note matters right now, so write it like a hook, not a title.

Then link generously. When one note references another, wire them together with [[wikilinks]]. Over a few months the brain stops being a list and becomes a graph — and the assistant can walk from a decision to the gotcha that caused it to the best practice that came out of it.

The part nobody warns you about: coexistence

Here's the failure mode I hit early, and it's the whole reason this post exists. When more than one writer touches the same brain — you, plus an assistant, plus the background agents you'll inevitably add — the naive "let the AI update the notes" approach turns into the AI quietly rewriting your notes. I watched an agent "tidy up" a section I'd hand-written and lose a nuance I cared about. It wasn't malicious. It was just doing what I'd have done to my own draft. That's the problem: an agent editing shared memory like it's its own scratchpad will step on the humans.

The fix is a short contract that every writer — human or agent — follows:

• Append, don't rewrite. Add to the bottom of the relevant section. Never delete the human's text without being told to.
• Date and attribute. Prefix entries with the date, and tag agent-written lines so authorship is never ambiguous. When something's wrong later, you can see who wrote it and when.
• Search before you create. Look for an existing note on the topic and update that, rather than spawning a near-duplicate. Duplication is how a brain becomes a junk drawer.
• Secrets by reference only. Never write a token or password into the brain — write where it lives ("in the password manager," "env var X"). The brain is readable by a lot of eyes and processes; treat it that way.

None of these are clever. All of them are load-bearing. The contract is what lets my hand-written notes and a machine's appended memories sit in the same file without either corrupting the other.

Trust the brain, but verify before you act on it

One more discipline, and it's a governance one. A recalled memory reflects what was true when it was written, not necessarily now. If a note tells the assistant "the flag is called X" or "that logic lives in file Y," the right move is to confirm X and Y still exist before acting on them — the brain is a strong prior, not a live source of truth. I treat recalled notes as background context the assistant considers, never as instructions it obeys. That distinction matters more as you add agents: a memory store that agents blindly execute is a prompt-injection surface waiting to happen. A memory store they consult and verify is just a very good set of notes.

What it actually buys you

The obvious win is that I stopped re-priming every session — the cold-start tax is gone, and that alone paid for the build. The bigger win snuck up on me: the brain compounds. Every gotcha I hit once is caught the next time. Every decision I make is there to contradict me when I drift from it three weeks later. The assistant got measurably more useful not because the model got smarter, but because it stopped operating on a blank slate every morning.

If you're leaning on an AI assistant for real work, this is the highest-leverage thing you can build around it — and you can start with a single folder and one index file today. Write down the decisions, the gotchas, and the "how does this actually work" facts as you go, and give the assistant permission to do the same under a contract that protects your own notes. If you've built something like this and solved the multi-writer problem differently, I'd genuinely love to compare notes. As always, I'm here to help!

Every major technological shift begins with a distraction. Early motorists were captivated by engines, not the quiet discipline of the assembly line. Early computer users marveled at silicon, not the operating systems that made it usable. And today, in the early days of AI, we are all fixated on the models themselves — the power, the novelty, the spectacle. It feels like magic.

But the real transformation, just as before, is happening somewhere quieter. It is happening in the processes we build around the intelligence. The breakthrough is not the model. It is the repeatability.

I did not get there from a keynote. I got there by building one of these loops myself, watching it fail in ways I had not predicted, and slowly realizing that nearly every failure was a process failure wearing a model failure's clothes.

Are you still in the artisanal phase?

Right now, most of us interact with AI the way craftsmen once built cars: one at a time, by hand, improvising through each task. A prompt here, a tweak there, a little more trial and error. It works, but only in the way a single artisan produces a single beautiful object. It does not scale, it does not compound, and it certainly does not create reliability.

If you have ever fixed a bad output by adding another paragraph to a prompt that was already too long, you know exactly what I mean. I did that for months. The prompt grew, the failures got weirder, and every fix made the next problem harder to see.

The shift out of that phase is not a slow, decade-long evolution — it is happening right now, and every month brings new capabilities and new pressure to turn experimentation into something operational. The organizations that succeed will be the ones that treat process definition as the foundation of their AI strategy, not as documentation and not as an afterthought.

What changes when the workflow becomes the context

When you define a workflow, you are not just telling the AI what to do. You are building the rails it runs on.

This is where workflow graphs matter — not as diagrams, but as living systems. A workflow graph is a sequence of steps, each with a clear purpose, its own inputs and outputs, and its own guardrails. Instead of pouring every instruction and every scrap of context into a single prompt and hoping the model sorts it out, each step receives exactly what it needs. The workflow itself becomes the memory. The workflow becomes the context. The workflow becomes the intelligence that surrounds the intelligence.

Mine is a plan-implement-review loop built on LangGraph, pointed at issues in my own Forgejo instance.^[1] Each node gets a prompt assembled from the issue, not from a monologue I maintain by hand. That one change — generating the prompt from the workflow definition instead of curating it — did more for reliability than any model upgrade I made in the same period.

Why does a graph make failure easier to find?

Here is the part you feel immediately.

When something goes wrong in a single-prompt system, the failure is buried inside a thousand tokens of conversation and you get to go spelunking. In a graph, it is isolated to a step. A node can recognize that it is confused, classify the type of failure, retry with adjusted parameters, escalate to a different model, or hand the task back to a human with a clear explanation of what happened.

I will give you a real one, because the abstract version sounds tidier than it is. My poller had a re-arm path for a failed step that was effectively unbounded — a failure would re-arm, run, fail, and re-arm again. The loop looked busy and productive from the outside. What it was actually doing was burning tokens like tires doing donuts — plenty of smoke, plenty of noise, and the car never left the parking lot. The fix was not a better prompt or a smarter model; it was bounding the retry at the step that owned it. That bug was only findable *because* the step had an owner. In a single-prompt system it would have read as "the model is being flaky today," and I would have gone looking in entirely the wrong place.

This is how real automation behaves. This is how factories behave. And this is how AI becomes something you can actually trust with work you care about.

Which model should each step use?

One of the most liberating outcomes of this approach is that choosing a model stops being a guessing game.

In the artisanal phase, picking a model feels like picking a wine. Everyone has an opinion, nobody agrees, and half the time you are just hoping. Personally, I would rather choose a good whisky — at least there I know what I like.

In a workflow-driven system the choice gets boring, which is the highest compliment I can pay it. The process dictates the worker:

Anthropic makes the same distinction in their write-up on building effective agents — a workflow is a system where steps are orchestrated through predefined paths, and an agent is one that directs its own process.^[2] Most of what people call an agent works better as a workflow, and the reason is exactly this: predefined paths are the thing you can debug. I keep the routing itself outside the agent entirely, in a separate service that owns what models exist and which one a given request should land on — so changing my mind about a model is a routing change, not a code change in eleven places.

The part that still surprises me

You do not have to design these workflows from scratch.

The process of defining the process is also something the models are good at. They will draft the initial graph, propose improvements, spot the step you forgot, find the circular dependency you talked yourself into, suggest error-handling strategies, generate test cases, and refine the whole thing over time. I have had a model catch a missing failure path in a graph I had already stared at for an hour.

I have written before about why routing beats picking a favorite model, and about gating the steps that cannot be undone. Both are the same idea as this post, arrived at from different directions: the structure around the model is where the leverage is.

That is the blueprint phase — the moment when the foundation is poured, the scaffolding goes up, and the shape of the thing becomes visible. We are not replacing engineers, analysts, or domain experts; their judgment is what makes any of this worth automating. What we are replacing is the wasted time spent reinventing the same steps and solving the same problems from scratch, one bespoke task at a time. The people stay. The inefficiency goes.

Once your workflows are defined, you stop doing the repetitive work — the system does it. Your role shifts from execution to orchestration, from performing tasks to improving the machinery that performs them. That is the same shift that powered the industrial revolution and the computing revolution: from craft to engineering, from improvisation to repeatability, from isolated effort to systems that scale.

Where to start this week

If you want one concrete move, make it this: take the prompt you have been growing for months and ask what its three steps actually are. Not five, not twenty. Three. Give each one its own inputs, its own success condition, and its own failure behavior. In my own graph, two of the first three steps I wrote had no answer to "what should happen when this goes wrong" — and both of them turned into bugs later. That is where I would look first.

The key takeaway is this: AI is not new, automation is not new, but the combination — intelligence flowing through well-designed processes — is what makes the difference between a demo and something you can run on a Tuesday. The assembly lines of the future are not made of steel and conveyor belts. They are made of workflows, graphs, and the quiet confidence that comes from knowing the system will do the right thing because you designed it to.

And just like the great transformations before it, the real magic is not in the parts. It is in the process.

If you go decompose a prompt after reading this and find something ugly in step two, I'd genuinely like to hear about it — that's the most useful thing that can come out of a post like this. I'm here to help!

Sources

1. langchain-ai.github.io — langgraph
2. anthropic.com — engineering/building-effective-agents

Everyone sells the medallion architecture on the same promise: your Silver layer is cleaner than your Bronze. Deduplicated, conformed, typed, joined. That's true, and it's also not the payoff that matters. After building a governed memory store on Databricks — Delta for the raw tier, Postgres for the serving tier — I've landed on a sharper way to say what you're actually buying. It isn't a better Silver. It's a defensible one: a layer you can prove wasn't tampered with, that won't leak one client's existence to another, and where a cross-tenant write is a bug you literally cannot commit. Let me show you the difference, including the part where I was wrong.

The immutability I claimed but didn't have

I built a Bronze tier and called it immutable, because it was append-only. On Delta that's one property:

ALTER TABLE bronze.events SET TBLPROPERTIES ('delta.appendOnly' = 'true');
-- now an UPDATE, DELETE, or MERGE...WHEN MATCHED fails:
-- [DELTA_CANNOT_MODIFY_APPEND_ONLY]

On Postgres it's a BEFORE UPDATE OR DELETE trigger that raises an exception. Same idea, same good feeling: the raw archive can't be mutated. I shipped it. Then I ran the one test I should have run first — can the person the guardrail is meant to constrain turn it off?

ALTER TABLE bronze.events SET TBLPROPERTIES ('delta.appendOnly' = 'false');
DELETE FROM bronze.events WHERE embarrassing = true;
ALTER TABLE bronze.events SET TBLPROPERTIES ('delta.appendOnly' = 'true');

The delete succeeded. Of course it did. Anyone with ALTER can disarm the property, mutate the table, and re-arm it. The Postgres trigger has the identical hole — the table owner just runs ALTER TABLE ... DISABLE TRIGGER ALL and walks right in. I'll admit it plainly: I had this exact hole in shipped code, because the application's own write role owned the table. The guardrail was self-disarming, and the principal I was "protecting against" was the app itself.

Here's the lesson worth more than the code: append-only enforcement is a guardrail, not a control. A guardrail keeps you from falling off by accident. A control stops someone who's trying. Don't confuse the two — I did.

What actually prevents it: split the ownership

Prevention doesn't come from a smarter trigger. It comes from making sure the writer doesn't own the table. Separate the role that runs migrations and owns the schema from the runtime role that only inserts:

-- owner role runs DDL/migrations and owns bronze.events
-- runtime role can insert, and nothing else
REVOKE UPDATE, DELETE ON bronze.events FROM app_writer;
-- app_writer now has no ALTER, so it can't disable its own immutability

Now the app can append and cannot mutate — not because a trigger says no, but because the engine never granted it the power to say yes. Prevention comes from grants, not from a guardrail the writer can disarm. Treat append-only as defense-in-depth and tamper-detection. Never as the boundary.

Delta's real edge: it records the cover-up

So append-only can be turned off on any engine. Does Delta buy you anything here? Yes — and it's the single most underrated thing in the format. When someone disarms, deletes, and re-arms, every one of those steps is a numbered commit in the transaction log. Walk the history of a tampered table and you'll see the whole confession, something like:

version 4  UNSET TBLPROPERTIES (delta.appendOnly)
version 5  SET   TBLPROPERTIES (delta.appendOnly = false)
version 6  DELETE
version 7  SET   TBLPROPERTIES (delta.appendOnly = true)

The attacker put the property back exactly the way they found it. The table looks pristine. The log says otherwise, permanently. That's tamper-evidence — Delta doesn't stop the tampering, it makes the tampering undeniable. Plain Postgres gives you nothing equivalent without wiring up an audit trigger or logical decoding yourself. So the honest framing is: append-only is the guardrail, ownership is the control, and on Delta the transaction log is the auditor who never blinks.

The leak nobody checks: your catalog is telling on you

Now the tier that actually serves queries. If you isolate tenants with a schema each in a shared Postgres database, you've solved the data leak — client A's role can't read client B's rows. You probably haven't solved the metadata leak. Give client A's role zero data access to client B's schema, then run this as that role:

SELECT tablename FROM pg_tables WHERE schemaname = 'client_b';
-- ('memory')

Client A just learned that client B exists, and what their tables are called. pg_catalog is world-readable to anyone who can connect. For a hobby project, nbd (no big deal). For a multi-tenant design, that's a disclosure problem — the tenant roster is supposed to be confidential, and the database is handing it out.

This is where Unity Catalog earns its keep, and it's not the reason vendors usually pitch it. UC filters metadata by grant. A principal granted one catalog sees only that catalog in SHOW CATALOGS and in information_schema — the others don't exist as far as it can tell. Two rules make it hold:

• Grant USE CATALOG per tenant, and only per tenant.
• Don't hand out BROWSE broadly. It's a real Unity Catalog privilege that exposes an object's metadata — names, comments, tags — with no data access, and Databricks actually recommends granting it on catalogs to the All account users group so data is discoverable org-wide. For a single company that's a feature; for tenant isolation it's the one grant that re-opens the leak you just closed. Scope discovery per tenant instead.

One caveat so you don't over-trust it: metadata filtering lives on the UC governance path. If a tenant reaches the underlying storage or a UC-registered Postgres by a direct connection that skips the metastore, UC isn't in that path and can't filter anything. Governance you can bypass is decoration. Know where your control plane actually sits.

Isolation belongs on the connection, not on a column

The last piece is the one I feel most strongly about. The tempting way to keep tenants apart in a shared table is a namespace column that the application sets on every write. Don't. A wrong namespace value looks exactly like a right one. Nothing rejects it, nothing logs it as suspicious, and the contamination is silent and undetectable after the fact — you find out when client A sees a fact that belongs to client B, and by then it's in the Silver layer.

Push the guarantee down to where code can't be wrong. If the worker connects as the tenant's own role to the tenant's own database, a cross-tenant write isn't a value you have to get right — it's an operation the engine refuses. You've turned a mistake anyone could make into a bug that cannot be written. The invariant to hold onto: no code path ever holds a connection that can see two tenants at once. Demote namespace to what it's good for — scoping within a tenant (owner, project), where a slip is embarrassing rather than a breach.

So what did Databricks actually buy you?

Not a cleaner Silver. A defensible one. Delta's transaction log means you can prove the raw tier wasn't quietly rewritten. Unity Catalog's grant-filtered metadata means one client can't enumerate another. And the database engine — Databricks or Postgres — buys you connection-level isolation that turns a whole class of contamination into an impossibility instead of a code review. The dedup and the typing are table stakes; every tool does that. Defensibility is the part you can put in front of a security review and a skeptical client, and that's the part that's worth paying for.

If you're running a multi-tenant medallion setup, go check one thing today: can the role that writes your Bronze also ALTER it? If yes, you have my old bug. And if you've found a cleaner pattern for any of this — especially metadata isolation on plain Postgres — I'd genuinely love to hear it. As always, I'm here to help!

A few weeks ago I stopped trusting my own traffic graphs. The edge dashboard told me one story — steady visitors, healthy numbers, nothing alarming. So I built a second view that reads the origin's own access logs instead of the edge's summary, and the story fell apart. When I split the traffic by who was actually making the requests, roughly 82% of it was automated. Scanners, scrapers, and probes. The humans were the minority on my own blog.

Here's the good news up front, because that number sounds worse than it turned out to be: once you can see the bots accurately, you can do something about them. I did. Let me walk you through how I measured it, what I found crawling around in there, and the layered defense I put up — most of which you can copy.

Why your traffic numbers are probably lying to you

Most of us read traffic from whatever the CDN or edge provider hands back. That's convenient, and it's also not origin truth. The edge has its own idea of what a "bot" is, it's tuned to sell you on how much it blocked, and it can't classify the actors the way you'd classify them if you had the raw requests in front of you.

I wanted the raw requests. My blog runs Ghost behind a tunnel, which introduces the first gotcha: at the origin, every request appears to come from the same internal proxy address. The real visitor IP isn't the connecting IP — it's tucked into a forwarded header. If you classify on the connecting IP, congratulations, you've decided 100% of your traffic comes from one machine. You have to resolve the real client from the header first, and then the picture gets honest.

So I shipped the reverse proxy's access logs into a log store and built a dashboard on top of them — a handful of panels that bucket every request into human, search engine, AI/LLM crawler, SEO tool, or scanner, per site. No SaaS, no black box. Just my logs, my rules for what counts as what. That's the part I'd push you toward: the value isn't the fancy dashboard, it's that you own the classification logic, so when a number looks weird you can go read the actual requests behind it.

What was actually knocking on the door

The breakdown, once I could see it: about 82% automated, roughly 864 human sessions, and around 88 AI/LLM crawlers over the window I looked at. The AI crawlers I mostly don't mind — that's the new SEO, and I'll take it. The 82% is the interesting part, and some of it is almost charming in how lazy it is.

• A single Wget/1.25.0 client hammering away at about 6,180 requests in 24 hours. Not subtle.
• A user-agent string that spelled it Mozlila — a bot faking "Mozilla" and misspelling the fake. Trust me on this one: if your attackers can't spell the browser they're pretending to be, you can afford to be aggressive with them.
• Named audit scanners and generic HTTP clients — the TLM-Audit-Scanner, pathscan, plain Go-http-client — walking the site looking for something to exploit.

The most telling signal wasn't a user-agent at all. It was what they asked for. My blog runs Ghost. So why is everyone requesting /wp-login.php, /xmlrpc.php, /.env, /.git, and /phpmyadmin? Those are WordPress and credential-leak paths that a Ghost site never serves. A human never types those. A real search crawler never asks for them. They're a perfect tell — a request for one of those paths is a confession.

Turning the tell into a trap

That confession is the hook for the whole defense, so let's dig into it. I set the blog up as a funnel, and each layer does one job.

Layer one is an intrusion-prevention system that parses those same access logs, subscribes to a community blocklist, and drives a plugin at the proxy to ban flagged IPs across the whole site. Self-hosted, auditable, no vendor holding the ban list hostage. This is the blunt instrument: known-bad IP, gone.

Layer two is a proof-of-work challenge in front of the blog. Think of it like a bouncer who makes you solve a small puzzle before you come in — trivial for a real browser, expensive at scale for a scraper firing thousands of requests. The key detail so you don't wreck your own SEO: allowlist the verified good crawlers (Google, Bing, DuckDuckGo) so they walk straight through. You want to tax the scrapers, not the search engines that send you readers.

Layer three is where it gets fun. Remember those confession paths? Requests for them get routed into a tarpit — a service whose entire job is to waste a bad bot's time. It answers slowly and hands back an endless maze of links that go nowhere. In my logs I can watch a probe for /wp-admin take 20 seconds to resolve, /.env take 31, /xmlrpc.php take 12. Every one of those seconds is a second the scraper isn't spending on someone else.

I'll admit I fumbled the tool choice here first. I reached for one popular tarpit, went to read its docs, and found the author deliberately poisons his own site against AI scrapers so thoroughly that I couldn't get a clean read of the documentation to deploy it. Respect the commitment. I pivoted to a different engine and moved on.

Feeding the scrapers a fake version of me

Here's the part I'm quietly proud of. A tarpit that serves obvious garbage gets fingerprinted and skipped. So instead of random noise, I trained the maze on a Markov model built from about 3.4MB of my own published writing. Now the scrapers that harvest my domain get believable, on-brand, completely fake content — sentences that babble in my data-engineering voice about partitioned tables and granted access and daily backups, stitched into paragraphs I never wrote.

If someone is scraping the web to train a model or resell my content, they don't get my posts. They get a plausible hallucination of me. That's a better outcome than a 403, because a 403 tells them to try harder and the maze tells them nothing is wrong.

The layers also talk to each other. A bot that trips the tarpit or fails the challenge shows up in the logs, the intrusion-prevention layer notices the pattern, and the offending IP gets banned at layer one — so the next request from that address doesn't even reach the funnel. The trap feeds the wall.

What this actually buys you

Two things, and neither is "I blocked all the bots" — you won't, and chasing that is a treadmill. First, honest numbers. When 82% of your traffic is noise, every conversion rate, every "popular post," every engagement metric you've been reading is distorted by it. Cleaning the classification cleaned my analytics more than any tag-manager change ever did. Second, you stop subsidizing the people abusing you. Serving real pages to a scraper firing 6,000 requests a day costs you compute and bandwidth; sending it into a tarpit costs it instead.

One honest caveat before you run with this: the granular numbers here come from my own traffic capture over a single window, and I'd re-check them against the live dashboard before I quoted them in a boardroom. The shape of the finding is rock-solid; the second decimal place isn't the point.

If you run your own site, go build the origin-truth view before you build anything else — you can't defend against traffic you can't see accurately. And if you've found a cleaner way to classify actors from raw logs, or a tarpit engine you like better, I'd genuinely love to hear it. As always, I'm here to help!

I set out to build a memory layer for my agents — somewhere they could write what they learned and read it back later without me hand-feeding context into every prompt. A few weeks in, I looked at the schema I'd drawn and laughed, because I'd reinvented something I've been building in data lakes for years. Bronze, Silver, Gold. The medallion architecture, transplanted out of the warehouse and into an agent's recall. It turns out the pattern that keeps analytics honest is the same one that keeps memory honest.

If you've built medallion pipelines, you already know most of this — you just haven't pointed it at an LLM yet. Let me walk you through the transplant, layer by layer, and the one invariant that makes the whole thing work.

Bronze: raw, immutable, append-only

Every event an agent or a human emits lands first in a single table I think of as Bronze — an append-only archive. Raw content goes in verbatim and is never modified after insert. A session transcript, a note, a commit, an issue: it gets wrapped in a small contract and stored exactly as it arrived.

That contract carries a few fields that earn their place. A source and a source_event_id together form an idempotency key, so re-submitting the same event returns "duplicate" instead of creating a second copy — re-running an ingest is safe. A namespace scopes the record to an owner or a project and acts as a contamination firewall, so one client's memory can't bleed into another's. And a provenance_root points back to the upstream source of truth, so every memory can be traced to where it actually came from.

The discipline here is the same as in any lakehouse: Bronze is sacred and dumb. It doesn't interpret, it doesn't clean, it doesn't dedupe the content. It just remembers, faithfully, forever.

Silver: derived, cleaned, the part the system owns

Silver is where the raw event becomes usable knowledge, and it's owned entirely by the engine — no consumer reaches in and writes here. For each Bronze row, a derivation runs in four steps:

1. Distill. An LLM pulls atomic facts out of the raw content — each fact self-contained, specific, and testable. One messy transcript becomes a handful of clean assertions.
2. Embed. Each fact gets a vector, indexed for similarity search. This is what makes recall semantic instead of keyword-bound.
3. Resolve conflicts. Before a new fact is stored, the system finds existing facts that are close in vector space and asks an LLM to judge: is this new, a refinement, or a contradiction? It resolves to add, update, supersede, or skip.
4. Score. The fact gets an importance score, so later recall can prioritize what matters.

The supersede case is the one I want you to sit with. When a new fact contradicts an old one, the old fact isn't deleted — it gets an end-timestamp and steps aside. Yesterday's truth is still on disk, marked as no-longer-current. Memory doesn't overwrite; it keeps a history of what it used to believe and when that changed.

Gold: shaped for whoever's asking

Gold is what consumers actually touch, exposed through capability-gated tools — search, read, list — so an agent gets only the access it's been granted. Nothing queries the raw archive directly; everything comes through this projection.

Gold is also where humans get to disagree with the machine. A person can write an overlay — an edit, an annotation, a redaction — anchored to a span of a derived fact. On read, the system composes the Silver fact with the human's overlay laid on top, and the human's version wins. The machine's derivation is a draft; a person's correction is final. When Silver later re-derives from Bronze with a better model, those overlays re-anchor onto the new text rather than getting silently dropped. Human judgment survives a model upgrade.

The invariant that makes it all work

Here's the line that holds the whole structure up, and it's lifted straight from the lakehouse playbook: Silver is always rebuildable from Bronze.

Everything above the raw layer is disposable. If a better extraction model ships next quarter, I don't migrate Silver — I throw it away and re-derive it from Bronze. If I find a bug in the conflict-resolution logic, same move: fix the code, rebuild from the immutable source. The refined layers are a cache of my best current interpretation, not a second source of truth. There's exactly one source of truth, and it's the dumb append-only table at the bottom.

That invariant is also what makes the system safe to automate, which is the part that surprised me. Because re-deriving Silver is reversible, the engine can do it on its own. The only operations that need a human in the loop are the genuinely irreversible ones — destroying a Bronze record, or overwriting a person's overlay. Reversibility, not the model's confidence, draws the line between what the system does autonomously and what it asks permission for.

Where it costs you

I'm not going to sell you a free lunch. Medallion-for-memory has a real bill attached. Every Bronze row triggers an LLM distill and an embedding call, so ingest isn't cheap — you're paying model inference to turn raw events into facts, at the rate events arrive. The conflict-resolution threshold is a genuine knob, not a setting you get right once: too loose and contradictions slip through as duplicate "facts," too tight and real updates get filed as brand-new beliefs. And the rebuild-from-Bronze guarantee is only as good as your discipline about Bronze — the day something quietly prunes the raw layer to save space, the entire safety story collapses and you won't notice until you try to rebuild.

None of that has talked me out of it. The same separation that keeps a data lake trustworthy — immutable raw, disposable derivations, a consumer layer that never mutates the source — is exactly what an agent's memory needs, for exactly the same reasons. I've been doing medallion since before I called it that. It just took building a brain for my agents to realize the pattern was never really about analytics.

If you're storing what your agents learn, I'd love to know whether you landed somewhere similar or solved it a different way. As always, I'm here to help.

Most agent-authorization schemes I've seen ask the wrong question. They gate on confidence: is the model sure enough to act? Set a threshold, and if the score clears it, the agent does the thing. It feels rigorous. It's backwards. The axis that should decide whether an agent acts on its own isn't how confident it is — it's whether the action can be undone.

I landed on this building the memory layer for my agents, and it's quietly become the most reusable governance rule I have. Let me make the case, because once you see it you'll want to rip the confidence threshold out of your own system too.

Why confidence is the wrong gate

Confidence is the model's self-report, and self-reports are miscalibrated. You already know this — you've watched a model state something false in the same fluent, certain tone it uses for the truth. So when you gate authorization on a confidence score, the actions you wave through most readily are the ones the model is most sure about, and a confident-but-wrong action is the single most dangerous thing in the building. You've built a gate that opens widest exactly when you can least afford it.

There's a deeper problem. Confidence tells you the probability the action is right. It tells you nothing about the cost if it's wrong. And those are completely different questions. "Probably correct" is cold comfort when the failure mode is "permanently deletes the source of truth."

The rule: gate on reversibility

Here's the principle I run instead. An agent may act autonomously only when the action is reversible. Anything irreversible requires a human token — regardless of confidence score.

In my memory system, that line is drawn at the raw layer. Everything lands first in an immutable, append-only store I call Bronze — verbatim, never modified after insert. Every refined fact downstream is derived from Bronze and is, by design, rebuildable from it. So I can hand the reconciler — the autonomous agent that maintains memory — a lot of rope, because almost everything it does is reversible:

• Re-derive a refined fact with a better model? Reversible — Bronze is still there, rebuild it again. The agent does this on its own.
• Decide two facts contradict and supersede one? It's a soft delete — the old row gets an end-timestamp, never a hard delete. Reversible. Autonomous.
• Score a memory's importance? Pure derivation. Reversible. Autonomous.

And the short list of things it may never do without a human holding the token:

• Hard-delete a Bronze record. That's the one truly irreversible act in the system — you can't rebuild from a source you destroyed.
• Mutate a human's overlay — the edits a person made by hand. Those win over the machine, always, and overwriting them can't be cleanly undone.

Notice what's missing from that gate: the confidence score. The reconciler could be 99% certain a Bronze row is junk. It still can't delete it. Certainty was never the question — reversibility was.

Why this works

Here's the part I like: once you say it out loud, the logic is simple. Reversible actions are cheap to be wrong about, because being wrong costs you a rollback. Irreversible actions are expensive to be wrong about, because being wrong costs you the thing itself. So you put the human exactly where the cost of a mistake is unrecoverable, and you let the machine run everywhere the cost is just "do it again." You're allocating the scarcest resource you have — human attention — to the only place it actually has to be.

It also sidesteps the calibration problem entirely. I don't have to trust the model's estimate of its own correctness, which I shouldn't anyway. I only have to classify the action: can this be undone, yes or no? That's a property of the operation, not of the model's mood. It's far easier to get right, and it doesn't drift when you swap models.

It generalizes past memory

The reason I keep coming back to this rule is that it isn't really about memory. It's a governance primitive. The same shape shows up everywhere I let an agent touch something that matters.

In my infrastructure code, the stateful resources carry a prevent_destroy flag — automation can change almost anything, but it cannot delete the database volume without a human deliberately removing the guard. Same rule, different domain: reversible changes are automated, the one irreversible action needs a person. Once you start looking, the question "is this reversible?" turns out to be the right authorization boundary for nearly every autonomous system, not just the one that remembers things.

The catch

One honest caveat, because this rule has a load-bearing assumption: "reversible" has to actually be true. My whole gate rests on the invariant that the raw layer is immutable and everything else is rebuildable from it. The day that invariant quietly breaks — a "cleanup" job that prunes Bronze, a derivation that can't actually be re-run — every "reversible" action I waved through becomes irreversible after the fact, and the gate was a comfortable lie the whole time. So if you adopt this, the immutability of your ground truth isn't a nice-to-have. It's the thing the entire authorization model is standing on. Guard it accordingly.

Stop asking your agents how sure they are. Start asking what happens if they're wrong. If you've drawn this line somewhere different in your own system, I'd genuinely like to hear where — this is one I'm still sharpening. As always, I'm here to help.

I ran git shortlog -sne on my orchestrator repo last week, the way you do before writing a release note and wanting to thank the right people. I scrolled to the top expecting my own name. It wasn't there. The top committer is forgeai-dev. Second place goes to lthorn — another one of my agents. I show up third and fourth, because at some point I managed to give myself two git identities across two machines.

Here's the leaderboard, near enough as it sits in the repo:

   188  forgeai-dev    <forgeai-dev@toyboxcreations.net>
   181  lthorn         <lthorn@toyboxcreations.net>
   125  slowder        <slowder@toyboxcreations.net>
    97  Shannon Lowder <slowder@toyboxcreations.net>
    67  Claude         <claude@anthropic.com>

Two of my agents on top. Me in third and fourth — and if you count the Claude identity that signs AI-authored commits, arguably fifth too. If you'd told me three years ago that the most prolific contributor to my flagship project would be a process, not a person, I'd have assumed I'd lost control of the repo. It's the opposite. This is what it looks like when the loop is working — and, more to the point, when you can still tell who did what.

That's not one bot. It's a team of personas.

Let me be precise, because "a bot wrote my code" gets oversold. Those two top entries aren't one agent wearing two hats — they're two different roles, and the split is the whole point.

Early on, I ran what amounted to a single agent: Lorin Thorn — lthorn — my chief of staff. Everything routed through Lorin. Planning, coding, reviewing, all of it. That's why lthorn sits near the top of the list: for months it was the only worker, so every commit carried its name. Convenient, and a little bit of a lie — because "Lorin did it" told me nothing about which part of the process actually produced the change.

So I started taking Lorin apart. forgeai-dev is the dedicated developer persona — it owns the straightforward implementation work, which is why it has already overtaken Lorin at the top. A code-reviewer persona is taking over the LLM-based review pass. And Lorin is moving up, not out — into a PM and scrum-master role, running my projects, holding them to my delivery standards, and only pulling me in when a project as a whole is in trouble. As the workflows demand it, I'll add more: an analyst, whatever the next bottleneck calls for. I'm not building one clever bot. I'm building a small SDLC team and giving each role its own identity.

Why split one agent into many? Provenance.

Here's the part I care about most, and it's the reason the overhead is worth it. When every persona commits under its own name, inside a known workflow, at a known node, I can ask a question a single catchall identity can never answer: who made this change — which role, in which workflow, at which step? "Lorin did everything" is a black box. "forgeai-dev wrote it, the code-reviewer persona signed off, at these nodes in this run" is an audit trail.

And that trail is about to get deeper. Each persona is getting its own persistent memory — so the reviewer remembers the standards it has enforced, the developer remembers the patterns it has settled on, and nobody shows up to every run a fresh amnesiac. Identity, plus memory, plus a known position in the workflow, is what turns "the bot did it" into something I can actually inspect.

That matters because of a line I'd tattoo on the inside of every automation enthusiast's eyelids: automation of any kind, without auditability and provenance, is one mistake away from disaster. A loop that can land 188 good commits can land 188 bad ones just as fast — and if every one is signed "the bot," you will never find the single change that took you down. Provenance isn't paperwork. It's the difference between an autonomous system you can operate and one quietly accruing risk you can't see until it detonates.

Why I have two names in my own git history

You probably caught the thing I was hoping you'd catch: slowder and Shannon Lowder are the same person — same email, different display name. That's 125 and 97 commits that should be a single 222, split because I once committed from a machine where I'd never run git config user.name, and the default stuck.

It's a small mess, and a telling one. The agents commit under a stable, configured identity every single time. I'm the one who shows up under two names. When you start sharing a commit history with a process, the process is the disciplined one and you're the source of the entropy. That alone reordered my sense of who needs the guardrails around here.

What changes when a process is your top contributor

The leaderboard isn't a vanity metric. It changed how I work, concretely:

• I review more than I write. My highest-leverage hours stopped being "implement the thing" and became "read what the persona implemented and decide if it ships." Different muscle — the one worth building.
• Issues became the interface. The agents work off Forgejo issues, so a well-written issue is now a unit of production. A vague issue earns me a vague PR. I write issues more carefully than I used to write code.
• The repo runs while I sleep. A chunk of those commits landed during hours I was nowhere near a keyboard. For a solo operation, that's the entire game — leverage that doesn't need me in the room.

Where it bites

I won't hand you a clean story, because it isn't one. A persona that commits 188 times will commit 188 times' worth of mistakes if you let it, and it'll do it fast. Three things had to be true before this was a net win. The review gate is non-negotiable — every agent PR gets read before merge, by me or by the reviewer persona. The per-persona identity is what makes a regression findable — when something breaks, "who wrote this" needs a real answer, and "the bot" isn't one. And the line on what an agent may do without me has to be drawn at what's reversible, not at how confident the model claims to be.

The bot being my top committer was never a story about replacing myself. It's a story about changing the job — from the person who writes the code to the person who decides what good looks like, and who can still trace every line back to the role that produced it. The typing is increasingly not mine. The judgment, and the audit trail, still are. Those are the parts I'd never automate away.

If you're running an autonomous dev loop, I'd love to know what your git shortlog says — and whether you can still tell which of your agents did what. As always, I'm here to help.

I keep two completely different reflexes in my head for the same task, and last week they finally collided. The task is ordinary: an app needs a real database — proper transactions, low-latency reads and writes, not an analytics table playing dress-up. For years my hand went to the same place every time. Stand up a Postgres instance, wire it in, move on.

Watch which Postgres I reach for now, though — that's where the schism lives. If the work is 100% outside Databricks, on my own k3s cluster where I make the rules, I still hand-roll it: CloudNativePG, a Terraform module I keep around for exactly this, and every bit of toil that comes with owning your own database. Set the reclaim policy wrong and a stray terraform apply will cheerfully delete the data directory out from under you. I've done it. You learn fast to pin prevent_destroy = true on anything you'd cry over losing.

Then the database moves inside the Databricks ecosystem, and I don't do any of that anymore — I spin up a Lakebase instance instead. Same Postgres engine, none of the babysitting, and, more to the point, none of the seam I used to build between the database and the lakehouse. That's the consolidation this post is about: between Lakeflow and Lakebase, the pieces I used to stitch together by hand now ship as one platform. Let me walk you through what actually converged, and where it still bites.

The stack you used to stitch together

Think back to a "normal" data platform two or three years ago. You probably ran at least four moving parts: an ingestion tool to land raw data, a transformation-and-orchestration layer to shape it, a separate OLTP database to serve your application, and a warehouse to answer analytical questions. Four tools, four bills, four places for a credential to leak.

The failure mode wasn't any one of those tools. It was the glue between them. Every seam needed a sync job, and every sync job drifted. Your app database said one thing, your gold table said another, and you spent your Tuesday morning explaining to a stakeholder why the dashboard didn't match the product. I've debugged that exact mismatch more times than I'd like to admit, and the root cause was almost always the same: two sources of truth that were supposed to agree and quietly didn't.

Databricks spent the last two years collapsing those seams from both ends. Lakeflow came at it from the pipeline side. Lakebase came at it from the operational side. Put them together and the integration tax I described above largely disappears.

Lakeflow folded the pipeline into one surface

I wrote about Lakeflow when it went GA — go back and read that one if you want the full tour. The short version: Lakeflow unified the three things data engineers used to buy separately. Lakeflow Connect handles managed ingestion from your operational sources. Lakeflow Declarative Pipelines (the evolution of Delta Live Tables) handle transformation. Lakeflow Jobs (formerly Workflows) handle orchestration. One surface, ingest to gold.

What that buys you is fewer handoffs. Here's a declarative pipeline that lands rebel-fleet telemetry and rolls it up — notice there's no separate scheduler config, no airflow DAG, no glue:

-- Bronze: raw, append-only, straight off the source
CREATE OR REFRESH STREAMING TABLE fleet_events_raw
AS SELECT * FROM STREAM read_files('/volumes/telemetry/fleet/', format => 'json');

-- Silver: cleaned, typed, deduplicated
CREATE OR REFRESH STREAMING TABLE fleet_events
AS SELECT
     ship_id,
     CAST(event_ts AS TIMESTAMP) AS event_ts,
     status,
     sector
   FROM STREAM fleet_events_raw
   WHERE ship_id IS NOT NULL;

-- Gold: the question the commander actually asks
CREATE OR REFRESH MATERIALIZED VIEW ships_active_by_sector
AS SELECT sector, COUNT(DISTINCT ship_id) AS active_ships
   FROM fleet_events
   WHERE status = 'active'
   GROUP BY sector;

The pipeline declares its dependencies and Lakeflow figures out the order, the incremental refresh, and the backfill. You stopped writing orchestration code; you started declaring intent. That's the right trade — but it's only half the consolidation.

Lakebase put the operational database inside the lakehouse

The other half is the piece that retired that second reflex. Lakebase is a managed Postgres — real OLTP, real low-latency reads and writes — that lives inside the lakehouse platform instead of beside it. It's built on open-source Postgres, so your app talks to it with the same drivers, the same SQL, the same pg tooling you already use. No new query dialect to learn.

The piece I keep coming back to is database branching. You can branch a Lakebase database the way you branch a git repo — spin up an isolated copy for a test run, throw it away when you're done. If you've ever been afraid to run a migration against anything resembling production, sit with that for a second. You can branch the database, run the migration on the branch, point your integration tests at it, and only promote if it's green.

And because it's inside the platform, your operational tables and your analytical tables answer to the same Unity Catalog. One governance model. One place where access lives. The sync job I used to write by reflex — the one keeping a bolted-on Postgres in step with the gold layer — is the thing the architecture now removes.

What consolidation actually changes for your pipeline design

So what do you do differently? A few concrete shifts:

• Stop designing around the seam. If your reference architecture still has an arrow labeled "reverse-ETL back to the app database," question it. When the operational store lives in the lakehouse, the round trip from gold table to serving layer collapses into a read.
• Treat your database like code. Branch it in CI. Run the schema change against a throwaway branch, test against real shapes of data, and promote on green. This is the workflow application engineers have had for years and data engineers mostly haven't.
• Govern once. Put your access policy in Unity Catalog and let it cover both the operational and analytical sides. Stop maintaining two ACL models that have to agree.
• Push transformation into declarations. If you're still hand-writing orchestration, you're maintaining code the platform will now maintain for you. Spend that attention on the logic, not the plumbing.

The through-line: fewer integration points means fewer places to drift, and drift is where your Tuesday mornings go to die.

Where it still bites

I don't do vendor cheerleading, so let me name the gaps, because consolidation is never free.

First, this is more lock-in, full stop. Every seam you remove is a seam you can no longer swap out. When your ingestion, transformation, orchestration, serving database, and warehouse all carry one logo, your leverage at renewal time goes down. That's a real cost — price it in deliberately, the same way you'd price in the integration tax you're escaping.

Second, "managed Postgres" is not "every Postgres." If your app leans on an exotic extension or a very specific version, confirm it's supported before you bet the serving layer on it. Managed always means a curated subset.

Third, consolidation tempts you to put everything on the platform. Resist that. A high-throughput, latency-critical operational workload with its own scaling story may still belong on infrastructure you control. The win is removing the seams that hurt — not collapsing every system into one because you can.

My rule of thumb: consolidate the seams that cause drift, keep the boundaries that buy you real isolation or leverage. The architecture should follow the failure modes, not the marketing slide.

The stack is one thing now — design like it

For most of my career, "data engineering," "the app database," and "analytics" were three different jobs with three different tools and a pile of glue in between. That's the part that consolidated. Lakeflow took the pipeline; Lakebase took the operational store; Unity Catalog already had the governance. The seams I used to build by reflex are now design decisions I have to justify — and most of them I can't.

So go look at your own architecture diagram. Count the arrows that exist only to keep two systems in sync. Those are the ones on the chopping block in 2026. If you've already torn some of them out — or if you've hit a wall where the consolidation didn't deliver — I'd genuinely love to hear how it went. As always, I'm here to help.

But there's a gap neither of them touches, and it's the one that determines whether your agents actually produce useful results: the quality of the context you give them.

Agent Mesh tells you how to run agents. OpenSharing tells you how to share agent skills across organizations. Neither tells you how to manufacture context that makes those agents smart about your specific problem, in your specific environment, with your specific history. That's not a protocol problem. It's a memory problem.

The Garbage-In Problem for Agents

The fundamental failure mode I see in production agent deployments is not model capability — it's context quality. An agent reasoning about a pipeline failure has access to a generic system prompt, the immediate error message, and maybe a few recent logs if someone wired that up. It doesn't have the history of how this table has behaved over the last six months. It doesn't know that this exact error pattern appeared twice before and both times it was a schema evolution issue upstream. It doesn't have the context of what remediation worked last time.

That missing context isn't secret or hard to find. It exists in your pipeline run logs, your incident records, your agent's own previous interactions. The problem is that it's scattered across storage systems with no retrieval layer that understands what's relevant right now, for this specific task, weighted by how recent and reliable each piece of information is.

The result is an agent that reasons well but decides poorly, because it's reasoning from an impoverished context. The model isn't the bottleneck. The memory system is.

What Context Manufacturing Actually Requires

Retrieval is the obvious first answer, and it's necessary but not sufficient. A vector similarity search over historical data gets you semantically relevant documents. What it doesn't do:

• Weight by recency: a note from two years ago about how this table schema worked under a different ETL system is technically relevant but practically misleading. Context needs temporal decay.
• Fuse multiple signals: the best match under vector similarity isn't always the best match under keyword relevance. A hybrid retrieval that combines semantic search, full-text search, and a reranker produces better results than any single method.
• Shape for the consumer: a pipeline triage agent needs context in a different shape than a stakeholder report agent. Raw retrieved documents aren't the right unit; consumer-shaped views are.
• Improve over time: if the context you provided led to a bad agent decision, the memory system should learn from that — flagging the divergence, surfacing it for correction, tightening the retrieval on the next call.

This is what I'm building with Cortex Forge, and why I think of it as a context manufacturing system rather than just a vector database.

How Cortex Forge Approaches It

The architecture follows a medallion model with strict tier separation.

Bronze is the immutable, append-only archive — every raw event, run log, conversation turn, and note captured verbatim. Nothing is ever deleted from Bronze without a human-gated operation. It's the eidetic layer: the guarantee that nothing is lost.

Silver is the derived, system-owned knowledge layer. The engine processes Bronze events into structured notes and extracted facts — cleaned, deduplicated, reconciled. Silver is regenerable from Bronze, which means if the derivation logic improves, you can rebuild it without losing the source record. Human edits are captured as overlay patches — external authority over the system's internal notes, with the human winning on short-term disputes while the system's model of revealed behavior accumulates over time.

Gold is where retrieval happens. Consumer-shaped views over Silver: pgvector indexes for semantic search, BM25 indexes for full-text, per-agent memory sets scoped to specific workflows. A retrieval request against Gold runs hybrid search — HNSW vector + BM25 fused with Reciprocal Rank Fusion, passed through a reranker, filtered by temporal relevance with recency decay. The result isn't a list of documents — it's a ranked, weighted, consumer-shaped context optimized for the specific agent and task making the request.

The MCP server is the Gold consumer interface. Any MCP-compatible agent — LangGraph, Claude Code, Copilot, a custom agent behind OmniRoute — hits the same endpoint and gets back context shaped for its declared purpose.

The Reconciler Is What Makes It Self-Improving

The piece that differentiates this from a well-engineered vector store is the reconciler. Cortex Forge tracks the divergence between what the system believes to be true (Silver) and what is revealed through actual behavior (Bronze). When an agent's decision based on the manufactured context led to a correct outcome, that's a signal. When it didn't, that's a signal too.

The reconciler surfaces flagged divergences — "you stated X, but six months of behavior suggests Y" — for human review. The human's verdict is itself a Bronze event, feeding back into the accuracy of future Silver derivations. The system gets less wrong over time not through automated self-modification but through a structured human-in-the-loop feedback cycle that the system itself generates.

The governing rule is simple: autonomous action is permitted only for reversible operations. Destructive or irreversible operations — deleting a Bronze record, modifying a human overlay — require human authorization regardless of the system's confidence. Confidence affects ranking and whether to surface a proposal; it never authorizes irreversible action.

Where This Plugs Into Agent Mesh and OpenSharing

The Cortex Forge MCP server is itself an agent skill in the OpenSharing model. A provider that wants to offer enriched context retrieval — temporal-aware, hybrid-search, consumer-shaped — can publish the skill through OpenSharing's standard share/schema/asset hierarchy, with scoped credentials and zero-copy access. Any recipient who has been granted access can wire the MCP endpoint into their own agent stack without copying any underlying data.

For Azure Agent Mesh, the connection is even more direct. Register the Cortex Forge MCP server in Azure API Center alongside your other agent skills and tools. The API Center data plane MCP server makes it discoverable to any agent in the mesh. An agent running on Foundry Hosted Agents hits the Cortex Forge endpoint the same way it hits any other registered tool — through the unified discovery surface.

Both protocols were already designed to accommodate exactly this kind of infrastructure-as-a-skill. The MCP standard is the seam. Cortex Forge sits on the provider side of that seam, manufacturing context. The agent sits on the consumer side, using it.

The Practical Difference

I've been running agents with and without this kind of memory layer on the same tasks. The difference isn't subtle. An agent with access to a well-manufactured context from Cortex Forge makes better triage decisions on pipeline failures because it can reason about historical patterns, not just the immediate error. It catches recurrences that would otherwise look like new incidents. It proposes remediation approaches that worked before, rather than generating something from first principles.

The models are the same in both cases. The routing layer is the same. The only difference is whether the context going into the model call is generic or manufactured. That difference shows up in every decision the agent makes downstream.

Both OpenSharing and Azure Agent Mesh assume you've solved the context problem. Cortex Forge is my answer to that assumption. As always, I'm here to help if you're thinking through the memory layer for your own agent stack.

Unity AI Gateway, announced at DAIS this week, is the feature I've been waiting for since Agent Bricks shipped last year. It's a centralized governance layer for model access in Databricks — you configure which models are approved for use in your environment, who can call them, with what data access, at what cost budget, and with what logging requirements. Every model call in your Databricks environment goes through the Gateway.

For organizations that have been letting teams call foundation models from notebooks without any governance visibility, this is the compliance and cost control story you've been missing.

What the Gateway Actually Controls

Model allowlisting: your security team approves the set of models available in the environment. A team can't call an unapproved external model from a Databricks notebook once the Gateway is enforcing the allowlist.

Cost budgets: per-team or per-project token budgets with alerting when approaching the limit. The "who spent $40k on OpenAI calls last month" forensics conversation goes away when you have budget enforcement at the platform level.

Unified audit logging: every model call through the Gateway — model invoked, tokens consumed, user, timestamp, output classification if configured — lands in a Unity Catalog table. The same lineage and governance you have for your data applies to your model calls.

The Integration With Unity Catalog

The tightest part of the integration is the connection between Unity Catalog permissions and what data a model can be called with. A model call that includes data from a table the calling user doesn't have read access to can be blocked at the Gateway level. That's the data access governance story for AI that's been missing from every platform I've worked with. It's still early, but the architecture is right. I'm here to help design the Gateway policy structure for your environment.

The performance gap between open-weight models and closed frontier models has spent the last year collapsing faster than anyone predicted. Epoch AI's tracking puts open weights at roughly a three-to-four-month lag behind state-of-the-art closed models on average. For coding tasks, the gap has effectively closed — DeepSeek V3.2 and MiMo V2 Pro sit within striking distance of Opus 4.8 on real-world workloads. For complex reasoning, the closed frontier still holds a meaningful edge.

That remaining gap matters less than people think, for a reason that's easy to miss: the tasks in your pipeline are not uniformly hard. Most of them are nowhere near the frontier. And if you're routing every request to a frontier model because "it's the best," you're paying frontier prices for work that a well-prompted 7B model handles correctly — while also handing your data to a vendor you can't audit, on infrastructure you don't control.

The mature architecture isn't "pick the best model." It's build a routing layer.

What the Performance Landscape Actually Looks Like

The wave of MoE (mixture-of-experts) open-weight models that landed in the first half of this year changed the economics more than any single benchmark result. Models like DeepSeek V4-Pro, Qwen 3.6-35B-A3B, and Mistral Small 4 achieve very high active-parameter efficiency — only a fraction of total parameters activate per token, which means they run fast on modest hardware while delivering quality that rivals much larger dense models.

The result is a bifurcated market. For routine tasks — classification, extraction, structured generation, code templating — open-weight models are now the volume leaders. For the hardest reasoning, long-context synthesis, and nuanced generation, the closed frontier still earns its premium. The right response to this landscape is not to pick a side. It's to route.

The Stack I'm Running

I've been building toward a multi-provider routing architecture, and after spending time testing different approaches, I landed on OmniRoute as the gateway layer. It's an OpenAI-compatible endpoint that routes across 200+ providers — closed APIs, local inference endpoints, everything — with 15 routing strategies, 4-tier auto-fallback, and a prompt compression pipeline that cuts token counts 15-75% per request before the model ever sees them.

The compression piece matters more than I initially gave it credit for. Fewer input tokens means lower cost at every provider, lower latency everywhere, and meaningfully better performance on smaller models that struggle under bloated prompts.

Behind OmniRoute I'm running about a dozen providers. The interesting one for this post is the local tier: models running on a Mac Mini via MLX and Ollama. MLX became the dominant Apple Silicon inference backend after Ollama switched its Metal backend to MLX — it's 30-60% faster than the previous llama.cpp approach, and 3-4x faster on prompt processing on M4 hardware. On a Mac Mini M4 Pro with 64GB unified memory, a MoE model like Qwen3-Coder-30B runs at around 130 tokens per second — fast enough for real pipeline work, not just demos.

That local tier covers three things nothing in the cloud can: zero per-token cost at any volume, full data sovereignty (the payload never leaves the machine), and offline operation when the cluster is down.

The Three-Tier Routing Model

The routing logic I've arrived at isn't complicated, but it has to be explicit. Here's the actual decision tree:

Local models (Mac Mini / self-hosted): Anything involving sensitive client data, high-volume routine tasks where per-token cost accumulates, anything that needs to work offline, and anything where I want absolute certainty the payload doesn't leave my network. These run at zero marginal cost and with complete sovereignty.

Mid-tier cloud (Mistral Small 4, DeepSeek V3.2, Haiku 4.5, open-weight providers): Tasks that need more quality than local models reliably deliver, but don't need frontier reasoning — complex extraction, multi-step code generation, structured analysis. Cost is a fraction of frontier, latency is acceptable, and quality meets the bar.

Frontier cloud: Reserved for tasks where the quality difference is real and worth paying for — complex multi-step reasoning, high-stakes decision points in agent pipelines, content where prose quality visibly matters. Maybe 5-10% of total request volume in a mature pipeline. Which specific frontier provider fills this slot at any given moment is, as it turns out, not something you can assume in advance.

The routing decision itself runs on a fast small model. Task classification is cheap; sending the wrong task to the wrong tier is expensive.

Provider Risk Is Not Theoretical

The argument for multi-provider routing used to feel like defensive engineering — sensible in principle but unlikely to matter in practice. Then the US government issued an export-control directive requiring Anthropic to immediately suspend access to Fable 5 and Mythos 5 for all foreign nationals, citing national security concerns over a reported jailbreak. Anthropic couldn't segment foreign nationals from US-based users across a hundreds-of-millions user base on same-day notice, so they pulled both models for everyone — US customers included.

Teams whose workflows depended on Fable 5 had no warning and no graceful fallback. Those who had been routing through a gateway with multiple frontier options configured fell over to the next available provider automatically, with no human intervention and no pipeline downtime.

The shutdown wasn't caused by a technical failure, a pricing decision, or a vendor relationship gone bad. It was a regulatory action that the vendor had no choice but to comply with. That's a category of risk that doesn't show up in SLA discussions or uptime metrics, and it's one that a single-provider dependency can't hedge against.

I'm not going to speculate on the merits of the directive or predict when or whether access is restored. What the situation makes undeniable is the architecture point: if your pipeline has a hard dependency on any specific closed model, you're exposed to every kind of availability risk that model's provider faces — technical, commercial, and regulatory. A routing layer with multiple frontier providers configured doesn't eliminate that exposure. It makes recovery automatic instead of manual.

The Sovereignty and Cost Math

Running this architecture for several months, the economics are instructive. The local tier absorbs most of the volume for high-frequency pipeline tasks. The mid-tier cloud handles the work that needs more quality than local provides. The frontier tier handles a small fraction of requests. The blended cost across all three tiers is dramatically lower than routing everything to a frontier API — and the data exposure surface is dramatically smaller because the sensitive volume stays on-prem.

The other dimension people undercount is exactly what the Fable situation illustrated: provider resilience. A routing architecture that runs across a dozen providers with auto-fallback absorbs outages, pricing changes, model deprecations, and regulatory shutdowns, without a pipeline change or an on-call incident.

What This Is Not

This is not a recommendation to avoid frontier models or to treat any specific provider as unreliable. The closed frontier produces genuinely better results on hard reasoning tasks, and a well-configured routing setup should absolutely include frontier options. The point is that which frontier provider fills that slot should be a routing decision, not an architectural dependency.

The goal is calibration: the right model for the right task, routed automatically, with fallback when something fails — for whatever reason. Build the router first. Then figure out where each tier actually earns its slot in your pipeline. As always, I'm here to help.

The central message from DAIS this week, delivered by Ali Ghodsi in the opening keynote, was direct: AI doesn't have an intelligence problem, it has a context problem. If your CFO can't get an AI system to explain why margins changed, that's not a model capability failure — it's a context gap. The model doesn't have the enterprise-specific data, semantics, and business context it needs to give a meaningful answer.

That framing explains the entire 2026 Databricks product roadmap.

Genie One and Genie Ontology

Genie One is positioned as a smart AI coworker that understands your data — natural language queries against your lakehouse that produce accurate, business-contextual answers rather than technically-correct-but-business-wrong SQL. The underlying technology is Genie Ontology: a continuously-learning semantic layer that maps business terms to their underlying data representations in your catalog.

The ontology piece is the hard part that previous natural language to SQL systems got wrong. Knowing that "revenue" means SUM(net_order_amount) from a specific table, with specific filtering for refunds, in your specific business context — that's not something a general model knows. Genie Ontology learns it from your data and your corrections over time.

LTAP: The Transactional-Analytical Convergence

The other major architectural announcement is LTAP — Lake Transaction and Analytical Processing — which brings transactional and analytical workloads together at the storage layer rather than requiring separate systems with ETL between them. Combined with Lakebase now GA, this is Databricks making a serious structural argument that the lakehouse should be the operational database too.

The implications for pipeline architecture are significant: if your operational data and analytical data live in the same governed store, the data movement pipelines between them are reduced to transformation pipelines. That simplifies a lot of architecture that currently exists only to bridge the operational/analytical divide. I'm here to help think through what that means for your specific architecture.

Two and a half years in, with a production knowledge system, a working orchestration layer, local inference running alongside cloud providers, and prompt hygiene enforced at the infrastructure level — I want to be direct about something the AI industry is not particularly incentivized to say clearly: building this has been hard, it has taken a lot of time, and it is still not something I would recommend to most engineers as a place to invest significant effort without understanding exactly what they're signing up for.

That's not pessimism. It's the honest accounting that I would want from anyone describing a technology investment of this magnitude.

What LLMs Actually Are

Language models are pattern-completion engines trained on text at massive scale. They are remarkably good at generating text that follows patterns similar to their training data. They are not reasoning systems in the way a human expert reasons — they do not maintain internal models of the world that they update as new information arrives, they do not have persistent memory across sessions by default, and they do not know when they are wrong.

Everything in my stack that makes LLMs useful for professional work is infrastructure that works around these properties: the knowledge system compensates for the lack of persistent memory; the orchestration layer compensates for the lack of reliable multi-step reasoning; the output verification compensates for the fact that models don't know when they're wrong; the provider routing compensates for the fact that no single model is best for all tasks.

The tools are powerful. The infrastructure required to make them reliable for professional use is significant. That's the honest framing.

The Pattern-Driven Development Connection

The part of this story that I find most interesting — and that doesn't get enough attention in the AI tooling conversation — is how much the LLM value proposition depends on having well-established patterns in the first place.

I've been writing about pattern-driven development for over a decade: metadata-driven pipelines, configuration-driven frameworks, template-based code generation. The core idea is that consistent patterns enable automation. LLMs extend that idea in a specific direction: if your work follows consistent, recognizable patterns, language models can assist with the pattern-following parts — generating the boilerplate, applying the conventions, producing the structural scaffolding — freeing human attention for the parts that require genuine judgment.

The implication: engineers who already work in highly pattern-consistent ways get more value from LLMs than engineers whose work is more ad hoc. If you've invested in framework design, in consistent naming conventions, in well-documented architectural decisions — all of that investment pays forward into LLM assistance quality. The knowledge base I built is essentially an explicit representation of patterns I had already developed implicitly. Making those patterns explicit also made them accessible to a model.

Who Is Ready for This

The audience for serious AI tooling investment, as of right now, is narrow. You need enough engineering depth to evaluate model output critically — to catch the confident wrong answers and the subtle logic errors that look correct but aren't. You need enough infrastructure comfort to build and maintain the surrounding systems without being blocked by the operational complexity. You need enough patience to invest in tooling that pays back over months rather than days.

You also need either deep pockets or the hardware you bought before GPU scarcity made local inference inaccessible. The engineers running capable local models today are largely those who acquired the hardware before the market moved against them. That window may or may not reopen.

The honest assessment: AI-assisted development, done at the level of investment I've described, is currently viable for a specific population of technically senior, infrastructure-comfortable, high-pain-tolerance engineers. The consumer-ready version — the one that works well without significant setup, without ongoing maintenance, without deep expertise in the tools being used — does not yet exist. It is probably coming. The trajectory of improvement is real. It is not here yet.

What Comes Next

The work continues. The knowledge system needs better automatic ingestion. The orchestration layer needs better error recovery. The provider routing needs more sophisticated cost awareness. The de-identification pipeline needs to handle more edge cases. None of these are done; all of them are improving.

The question I keep returning to is not "when will LLMs be good enough?" — they're already good enough for a significant fraction of the work I do. The question is "when will the infrastructure required to use them reliably be accessible enough that the investment makes sense for a broader population?" The answer to that question depends on how the tooling ecosystem matures, and the answer is not yet obvious.

I'll keep building, keep documenting, and keep being honest about what's working and what isn't. If you're on a similar path — building the infrastructure, not just using the models — I'd genuinely like to compare notes. As always, I'm here to help.