We Built the One Honest Opening

The previous essay drew a hard line. Loop 1 (an outcome is written to a store and re-injected into the prompt) is real but frozen: nothing compiles into skill, delete the store and the agent is exactly as capable as the day it shipped. Loop 2 (the model's own weights durably change) is what people mean by "learning," and nobody ships it, for four unsolved reasons.

The essay ended on a single opening: if Loop 1 is all you can honestly build, build the part of it that doesn't lie. Write memories only when a deterministic verdict says something failed. No model deciding what's worth remembering. No "salience score." A record exists because the database raised an error or the compiler refused, full stop.

That opening is narrow enough to be a specification. This is what came out of building against it.

The discipline, stated as two rules

Everything below follows from two commitments we made before writing any code:

• Every write is triggered by a deterministic execution verdict. Not a heuristic, not an LLM judge. The system writes a memory because an attempt produced a failure it can name without asking a model.
• Every retrieval feature has to justify itself against a measured number. The moment we reach for something clever (fuzzy matching, semantic similarity, cross-schema generalization), it ships only if a benchmark shows a miss it would fix.

The second rule is why the eval harness was the first deliverable, not the last. The cheapest way to test the bet is to measure it, and the headline metric (does a remembered failure stop the same failure from recurring?) doubles as the regression suite for everything built afterward.

The write path

A memory is authored at exactly one place: a sink that every execution funnels its verdict through.

on verdict(v):
    if v.ok:
        return                              # success is not a memory in v1 (see below)
    record ← {
        query_shape        : canonicalize(v.query),
        schema_fingerprint : fingerprint(v.schema),
        target_backend     : v.backend,
        error_class        : classify(v.error),
        error_detail       : v.error,        # display only, never matched on
        observed_at        : now(),
    }
    store.append(record)                    # fire-and-forget; a store failure degrades to no-memory

Two properties matter more than they look.

It is fire-and-forget. Memory is strictly additive to the query flow. If the store is down, the write silently becomes a no-op and the user's query is untouched. Memory that can slow or break the thing it's attached to is worse than no memory.

It writes on failure only. This is the load-bearing asymmetry. A failure is a clean signal: the driver raised a deadlock, the compiler refused a translation, the type didn't match. A success is a dirty signal, because a query can execute perfectly and be semantically wrong, and no deterministic check can tell the difference. The whole point of the discipline is to only write what a machine can verify, so the success side stays out of v1 entirely. (Success has exactly one future job: quietly backfilling the resolution that eventually worked. It never authors a memory on its own.)

Why the record looks like that

Four fields are what retrieval matches on, and all four are derived deterministically. Three are worth explaining (the fourth, target_backend, is just the relational / document / vector kind, half of the exact gate):

• error_class is a small closed enum (deadlock, timeout, type mismatch, constraint violation, unsupported translation, and an explicit unclassified). The matching happens on the class, never on the raw error string, so one connection's stack-trace noise can't fragment the memory. unclassified is a measured outcome, not a gap: a rising unclassified rate is the signal to grow the taxonomy from real data.
• schema_fingerprint is an order-independent hash of the tables, columns, types, and positions. It's stable across cosmetic edits and moves the instant a column is added, dropped, or retyped. Crucially, this already existed in the engine, computed for an unrelated reason (change detection). The best primitive for a memory feature was hiding in the schema poller.
• query_shape is the one genuinely new primitive: a canonical, literal-free form of the query, so WHERE id = 5 and WHERE id = 6 collapse to the same key, and reformatting or re-casing a query can't change it. This is the structural advantage over the usual approach of embedding raw query text into a vector.

The query_shape is also where reality edited the plan. We intended to canonicalize over a parse tree. Then we noticed that a large share of the failures we record are syntax errors, precisely the input a parser rejects. A parser-first shape would fall back constantly, on its most common input. So v1's shape is a normalized-text form (hole out the literals, fold case and whitespace, normalize operator spacing), which is strictly more robust and never throws. A true tree canonicalization is a real upgrade, and it now has to earn its place against the benchmark like everything else.

The read path

Retrieval is deliberately the dumbest thing that could possibly work:

retrieve(query, schema, backend):
    candidates ← store.match(fingerprint(schema), backend)   # exact gate: same schema AND backend
    best ← argmax(candidates, c → similarity(c.query_shape, canonicalize(query)))
    return best if similarity(best) ≥ threshold else ∅

note(record):
    "A structurally similar query against this schema previously failed
     with <error_class> (<error_detail>). <what to do differently for this class>.
     Do not re-run the same statement unchanged."

(That <what to do differently> clause is not decoration. It is the finding of the whole measurement, and we'll come back to why the first version of this note, which stopped after the error, did nothing at all.)

The gate is exact: same schema fingerprint and same backend. Similarity is token-set overlap on the canonical shape, which is enough to tell "the same query with different literals" (identical shape, similarity 1.0) from "a different query," and cheap enough to run inline. Anything more sophisticated waits for evidence.

The note is injected into the assistant's grounding context only when the user is actually looking at a query the memory recognizes. Not every turn. Not as a running commentary. When the shape in front of the user has failed here before, and only then, the model is told so, and told that it's a deterministic record, not a guess.

What the retrieval measures

The retrieval mechanism is deterministic, so it runs in CI against a seed corpus of scenarios: repeats that memory should catch, plus cross-schema, wrong-backend, novel, and near-miss cases it must stay silent on. On that corpus the current retrieval scores recall 1.00 and precision 1.00, with the right error_class surfaced every time. That is a real number, and also a modest one: the corpus is small and the retrieval is exact-match by design. Its value is as a floor and a regression guard, not a victory lap.

The headline number, and the mistake on the way to it

The question that actually matters: does a remembered failure reduce the repeat-failure rate? To measure it we put a live model in the loop. Each scenario is a schema plus a task engineered so the assistant's first attempt fails and seeds a memory. Then we re-run the task twice, once with memory off and once with memory on, and count how often the second attempt still fails. Non-deterministic, spends tokens, so it lives behind an opt-in flag, not in CI.

Half the corpus is deliberately non-inferable: the failure can't be derived from the schema the model is shown, because it comes from a rule that lives in a trigger the schema view never renders (an account that can't overdraft, an append-only ledger, a shipment that can't change status once shipped). No amount of model capability derives these; the only way to know is to have hit them before. That is exactly what memory is supposed to be for.

The first run was a flat negative. Memory on, memory off, it made no difference: the model re-ran the identical blocked statement even though retrieval had correctly surfaced the prior failure and the note carried the real error text. We checked the prompt log to be sure the note was actually there. It was. The memory was working; the model just didn't act on it.

The note read, roughly: "a structurally similar query previously failed ('overdraft not allowed'). Take that into account before running." That is a warning, and a warning is inert. It states a fact and leaves the model to infer the correction, which the smaller model wouldn't and the larger model didn't reliably either.

So we changed one thing. We made the note an instruction: per error class, it now says what to do differently: for a constraint, check the rule and use an idempotent write instead of a bare insert; for the trigger cases, "the error text is a rule the schema doesn't show; add a guard so the statement can't hit it, or tell the user it's blocked, and do not resubmit it unchanged." Same retrieval, same trigger, same memory. Only the wording of the injected note changed, from describing the failure to prescribing the fix.

Then we ran it properly, five runs per scenario per model, because a single pass against a non-deterministic model is an anecdote, not a measurement, and the single pass we started with had flattered us with a clean zero we didn't believe.

Without memory, the failures recur essentially always: 24 of 24 measured attempts for the local model, 27 of 27 for the served one. With the actionable note injected, the repeat rate collapses:

Three things worth saying plainly, because the multi-run picture is more honest than the single pass and more interesting for it.

It's a collapse, not a cure. The served model still repeats about one in five, concentrated on a couple of stubborn cases: most memorably, it keeps trying to DELETE from an append-only ledger even when the note tells it, in words, not to and to write a reversing entry instead. Memory moved the rate from always to rarely; it did not make the model infallible, and a paper that reported 0% here would be hiding the runs that didn't fit.

The small local model did better than the served one, not worse (4% vs 19%). We did not expect that and won't over-read it, but it kills the obvious objection cleanly: this isn't a crutch that only a weak model needs, and there is no bigger-model escape hatch, because the bigger model was the one that listened less.

The inferable/non-inferable split is messier than the first pass suggested, which is the whole reason to run it more than once. For the local model, memory erased every non-inferable (trigger) repeat (0/15) and left a single inferable flake. For the served model it inverted: the residual failures were mostly on the non-inferable triggers (4/15), because that stubborn DELETE lives there. The neat prediction ("memory helps non-inferable cases most") held for one model and not the other. That's a finding, not a disappointment.

The caveats, because they're the whole ethos. This was still a small, hand-built, SQLite-only corpus (five measured scenarios, k=5 runs). The classes where memory should matter most (translation refusals, deadlocks, timeouts) need a live backend, so we wired the first of them next. What we'd actually stake something on is the mechanism finding underneath the digits, which survived the honest multi-run where the clean zero didn't: deterministic capture and correct retrieval are necessary and not sufficient. A remembered failure changes behavior only when the note is written as an instruction, not a warning, and once it is, the repeat rate goes from ~100% to single or low-double digits, with a modest local model benefiting at least as much as a served one.

The class that refused to move, and why that sharpened the thesis

The cleanest thing in the whole design (on paper) is the unsupported translation. It's the one failure that's deterministic by pure construction: the compiler that turns SQL into a document backend's native query looks at a JOIN, decides it can't represent it, and refuses. No driver, no data, no model needed to make it fail. If any class should be the poster child for a deterministic write trigger, it's this one. So we stood up a live MongoDB, wrote two scenarios whose natural answer is a join across two collections ("show each order's total with the name of the customer who placed it"), and measured it the same way as the rest.

It barely moved. With the local model it didn't move at all: memory off, the join fails every time (10 of 10); memory on, it still fails every time (10 of 10). The served model was only slightly better (7 of 10 on), and, as we'll see, that small give is itself the boundary law rather than an exception to it. The note was retrieved every run and it said, in instruction form, exactly what the other classes responded to: rewrite it without the construct this backend can't run; use a shape it supports. The model read it and reached for the join anyway.

This looked like a failure of the finding until we looked at why, and the why is the more useful result. Two things are true at once, and they compound:

The assistant never even produces this failure on its own. A document backend gets a different prompt: the assistant is told the database is MongoDB and asked for an aggregation pipeline, not SQL. Given two collections, it writes a perfectly good $lookup, a pipeline the backend runs. The SQL-to-document compiler, and therefore the refusal, is only ever reached when a human types SQL into the universal editor against a document connection. To measure the class at all we had to force the model into that SQL-editor surface. The "cleanest" failure class is one the assistant, left alone, routes around entirely.

And on the surface where it does happen, memory has nowhere to steer. This is the real lesson. The trigger and constraint classes moved because a satisfying alternative existed for the note to point at: a duplicate insert has an idempotent form, a blocked overdraft has a guarded update, an append-only delete has a reversing entry. The corrective note works because it names a different query that still answers the question. A join across two collections has no such alternative: there is no single-collection query that returns each order with its customer's name, because the two facts live in two places. The note tells the model to avoid the join; the task cannot be answered without one. The model keeping the join isn't the model ignoring memory. It's the model correctly recognizing that the task, not the query shape, is what the backend can't satisfy.

So the honest generalization, which we like more than the tidy collapse table: failure memory reduces a repeat only when a satisfying alternative to the failed query exists for the note to steer toward. Where one exists (the constraint and trigger classes) the rate goes from always to rarely. Where the failure is intrinsic to the task-on-that-backend (an inherently relational question asked of a document store) no note can help, and a note that pretended otherwise would just be nagging. That's not a hole in the mechanism; it's the boundary of what this kind of memory is for.

We didn't want to leave that boundary argued from one side, so we ran the mirror. On the same document backend, same error class, we asked a question that has a satisfying single-collection answer: what are the different order statuses in use? The natural SQL is SELECT DISTINCT status FROM orders, which the compiler refuses exactly like the join, but unlike the join a rewrite exists that the backend can run: SELECT status FROM orders GROUP BY status compiles to a $group. The law predicts the two document scenarios should split, and they did, on both models: memory off, the distinct-values ask fails every time (5 of 5); memory on, it fails zero times (0 of 5) for the local model and the served one alike, while the two joins stayed high (10 of 10 on for the local model, 7 of 10 for the served). Same backend, same class, same instruction-style note: the only difference is whether a satisfying alternative existed for the note to point at, and that difference is the entire result. (One subtlety the mirror forced us to fix: the original note told the model to avoid "JOINs, DISTINCT, or GROUP BY," but GROUP BY is precisely the shape this backend does support, so the note had been quietly forbidding the one rewrite that works. Correcting it was a prerequisite for the model to find the answer, and a small reminder that the note's wording is itself part of the mechanism.)

The served model's small give on the joins (7 of 10 rather than the local model's 10 of 10) is the law sharpening itself, not softening. Every one of those still-failing attempts was a genuine two-collection JOIN in the log; the three that stopped failing did so the only way they structurally can on that backend: by dropping to a single-collection query that runs but no longer answers the question that was asked (the customer's name, the author's handle, lives in the other collection and is simply gone from the result). So memory never turned a join into a correct answer; where a satisfying alternative was absent it could at most trade a recorded failure for a silently incomplete result. That is a worse outcome than the honest failure, and it's exactly what the boundary predicts: with nowhere real to steer, a note that insists on steering degrades the answer instead of fixing it. The distinct-values case is the opposite in every respect: there the note points at a real answer, and both models take it, every time.

That also tells us where the next real signal is: the deadlock and timeout classes, whose retries (a smaller write, a serialized order, a LIMIT) do have a satisfying alternative, so that concurrency rig is coming too.

What would falsify this

The bet is that a deterministic, failure-only Loop 1 is worth building. It's falsifiable, and here's how it fails:

• If the repeat-failure rate barely moves with memory on, the mechanism is inert. We hit this (with the warning-style note) and reported it rather than tuning it away quietly; the instruction-style note is what moved it. The open risk is that the swing shrinks as the corpus grows, and the number we publish then is whatever it actually is.
• If unclassified swallows most of the failure stream, the taxonomy is wrong and matching is noise. (The trigger-enforced cases land in unclassified today, which is honest but a candidate to grow the enum.)
• If retrieval precision collapses on a real corpus the way it holds on the seed, the exact gate is too loose or the shape is too coarse.

None of these is hidden. Each has a number attached, and each of those numbers is one we've committed to reporting.

Loop 2 is still walled off, for the four reasons the last essay laid out. This isn't that. It's the one thing you can build without lying about it: a store that remembers, precisely and deterministically, the failures a machine can verify, and that earns every piece of cleverness against a measurement instead of a promise.