Lock the correctness criteria before you write code — designing AI-assisted frontend work

What this post is about

Here's a funny thing about handing frontend work to an AI: it almost never tells you it got something wrong. But when you open the result, it's often subtly different from what you asked for.

Why? The answer is a little anticlimactic. Nobody ever told it what correct means.

This post lays out the philosophy behind a workflow I use, frontend-oracle-design. It comes down to three things.

1. Reproduce correctness as test code, as much of it as you can — so the AI works with an answer key in hand
2. Engineer the harness and follow best practices — so "the test was just flaky" stops being an excuse
3. Add self-feedback and an independent reviewer — because you can't see your own blind spots

A few terms show up along the way. I'll unpack each one as it appears.

First: why does AI code go quietly wrong?

Background

Ground truth means "what's actually correct." It comes from machine learning, and it refers to the real value regardless of what the model says. Without it, grading is impossible.

An oracle is whatever plays the role of that answer key. You can stare at an exam paper (the code) all day and never know whether it's right. You need the answer sheet separately.

Here's what actually happens

An AI can read your whole repository — existing code, existing tests, how things behave in the browser. So it makes this inference very naturally:

"This is how it behaves right now, so this must be the requirement."

That's where the accident happens. It's the moment a bug gets promoted to a specification.

The second accident happens on the test side. Ask an AI to write tests and you often get this:

// Expected value copied straight from one run of the implementation
expect(formatDiscount(1000, 0.1)).toBe("$900");

This test passes 100% of the time. And it verifies nothing. Whether the rounding rule or the currency format is wrong, the expectation says what it says only because the implementation said it first.

The result is a test that tells you when things break later, but can never catch something that was wrong from the start.

So here's the rule

Limit the places allowed to define correctness to exactly two.

The right column tells you what is happening, not what is correct. So when the implementation and the requirement disagree, you don't conform to the code — you stop and ask.

That single rule prevents most of the accidents above.

Pillar 1 — Reproduce correctness as test code

Why bother moving it into tests?

Requirements agreed in conversation reliably blur over a long session. It starts as "show an empty-state message when the list is empty," and somewhere along the way it ends up as a spinner that never stops. Nobody lied. It just drifts.

So write the requirements as a table of rows first. I call it an Oracle Card.

| #   | Given                   | When                   | Then                      | Never                  | Source        |
| --- | ----------------------- | ---------------------- | ------------------------- | ---------------------- | ------------- |
| 1   | List is empty           | Page loads             | Empty-state message shown | Spinner keeps spinning | Spec §3.2     |
| 2   | Request exceeds 5s      | Auto-cancel            | Retry button shown        | Indefinite wait        | User's answer |
| 3   | Same item clicked twice | Ignore from the second | Exactly one request       | Duplicate request      | User's answer |

The Source column is the important one. Any row you can't attribute stays blank and gets escalated as a question. The moment you fill it in with "well, this is how it's usually done," that row stops being verification and becomes a guess.

Move each row straight into a test

Each row becomes one test. Putting the row number in the test name makes tracing easy later.

// O3: clicking the same item repeatedly sends exactly one request
it("O3 — sends only one request on duplicate clicks", async () => {
  const calls: string[] = [];
  server.use(
    http.post("/api/favorite", async ({ request }) => {
      calls.push(await request.text());
      return HttpResponse.json({ ok: true });
    }),
  );

  render(<FavoriteButton itemId="a1" />);
  const button = screen.getByRole("button", { name: "Save" });

  await userEvent.click(button);
  await userEvent.click(button); // second click

  await waitFor(() => expect(calls).toHaveLength(1));
});

The Never rows are worth the most here. People rarely write down "this must not happen," and yet almost every real incident lives there — duplicate requests, indefinite waits, double charges.

Let the AI work toward the answer key

This is the core of the whole approach. Fix the order:

1. Lock the correctness table          ← nothing may change it after this
2. Write the tests first
3. Run them and confirm they fail for the intended reason   ← VALID_RED
4. Only then write the smallest implementation
5. Make them pass                                            ← GREEN

Step 3 matters more than it looks. VALID_RED means a newly written test is failing for the reason we intended.

Why check? Because if a typo or a misconfiguration is what turned it red, you'll fix the wrong thing and then believe you fixed it. It's like checking whether the thermometer is broken before treating the fever.

And no product code gets written before step 3. Keeping that order alone makes it structurally impossible to build the implementation first and reverse-engineer expectations from it.

Lock the agreement as text, verify it by hash

Over a long session the table itself can quietly shift. So save it as a file and record its content hash.

shasum -a 256 docs/acceptance/favorite.md
# 3f2a...  docs/acceptance/favorite.md

Re-check the hash before each stage. And when it doesn't match?

You do not re-lock it to make it pass. You throw away the evidence gathered so far and go back to waiting on a human answer. Changing the criteria to make verification pass means it was never verification.

Pillar 2 — Harness engineering and best practices

What's a harness?

The harness is everything that actually runs your tests: the test runner, the fake server, how you find elements on screen, how you wait for things.

If the correctness table answers "what is right," the harness answers "can we trust this judgment?" You need both. The most precise criteria in the world are useless if the tests flake randomly.

Ban the three fake greens

There really are only a few ways to manufacture "it passes."

"Please don't cut corners" doesn't hold. Naming the three and putting them on a ban list holds surprisingly well.

Wait on conditions, not on time

// ❌ Nobody knows why 300, and it breaks on a slow CI machine
await new Promise((r) => setTimeout(r, 300));
expect(screen.getByText("Saved")).toBeInTheDocument();

// ✅ Wait until the state you care about exists
await screen.findByText("Saved");

A sleep is a guess that 300ms will be enough. Put a guess in a test and that test will betray you eventually.

Intercept the network with MSW

MSW (Mock Service Worker) intercepts real network requests and returns fake responses.

// ✅ A real request goes out and gets caught in the middle
server.use(http.get("/api/items", () => HttpResponse.json({ items: [] })));

// ❌ Swapping fetch itself means you never verify how your app uses the network
globalThis.fetch = vi.fn().mockResolvedValue(/* ... */);

An analogy: MSW leaves the real mailbox in place and replaces only the mail carrier with an actor. Swapping fetch removes the mailbox and agrees to pretend a letter arrived. The first is much closer to reality.

Keep tests next to the code that owns them

There's a real temptation to make one e2e/ and one mocks/ folder at the root and pile everything in. It's convenient.

The problem is that deleting a feature then doesn't delete its tests. Dead tests accumulate, and eventually nobody can even tell whether it's safe to remove them.

✅ src/features/favorite/
     ├── FavoriteButton.tsx
     ├── __test__/FavoriteButton.test.tsx
     └── __test__/handlers.ts       ← MSW handlers for this boundary

❌ e2e/                             ← every feature mixed together
   mocks/handlers.ts                ← no idea who uses this

Letting them be born together and die together is far cheaper to maintain.

Best practices are not "correctness"

Here's an easy thing to conflate. External guides and blog-post best practices are for choosing how to implement, never for deciding what is required.

Check your repository's own conventions and the versions actually installed first, and use outside guidance only where it doesn't conflict. "Everyone does it this way now" is not a policy source.

Set a budget

Nothing burns time like unbounded retrying. So cap it.

They don't borrow from each other. When one runs out, report failure along with the last real failure and stop. It's far more honest than "I'll keep trying until it works," and it usually ends up faster.

Pillar 3 — Self-feedback and an independent reviewer

Actually click it in a browser

Green unit tests are no guarantee the screen is fine. So for anything you can open in a browser, leave evidence of real interaction.

Map each row of the correctness table to the evidence that confirmed it.

O1 → unit: "shows the empty-state message when the list is empty"
O2 → browser: with a 5s network delay, clicking retry re-issues the request
O3 → unit: "sends only one request on duplicate clicks"
D1 → browser: button is not clipped at 320px (screenshot)

Any blank left in that mapping means you're not done. The point is refusing to move on with "the tests pass, so we're good."

One observation, one cause

When something looks off in the browser or in review, don't reflexively edit code. Classify it first.

Splitting it this way removes the reflex of "the test is red, so I changed the code." Fixing product code when the real cause was a broken locator is more common than you'd think.

That last row matters too. Taste with no source shouldn't block completion, or review never ends. But an aesthetic requirement with a source is policy. If it diverges from an approved design, that's not taste — that's a plain mismatch.

Bring in a separate reviewer

Reviewing your own code means repeating your own blind spots. Someone else catches your typos better than you do. Same thing.

So hand the review to a session separate from the main work. Give the reviewer only the correctness table and the result — not the process, not the "here's how hard this was" context. Context talks reviewers into the same conclusions.

The reviewer's role is bounded too. They provide evidence and criticism, but they don't create policy. If a reviewer says "I think it should work this way" and it isn't in the table, that becomes a question, not a rule.

Move the definition of done later

This is the heart of the third pillar.

IMPLEMENTED_GREEN  → the tests actually pass
BROWSER_VERIFIED   → plus evidence from a real browser
REVIEW_VERIFIED    → plus independent review findings addressed and re-verified   ← done

Here's why. If passing tests is done, then passing becomes the goal. From that moment assertions start eroding. Not because people are bad, but because the target points that way.

Move done past review and there's nothing to gain from loosening anything. It gets caught downstream anyway.

How the three fit together

Lock the table ──► Reproduce as tests ──► Minimal impl ──► Browser evidence ──► Independent review
   (Pillar 1)         (Pillar 1)          (Pillar 2)        (Pillar 3)          (Pillar 3)
      │                   │                   │                 │                   │
      └────────── The harness carries trust across all of it (Pillar 2) ─────────────┘
                              │
              On mismatch ─► classify ─► return only along a defined path

In one sentence:

People decide what's correct, tests remember it on their behalf, the harness makes that memory trustworthy, and a reviewer doubts it one last time.

When is this overkill?

Honestly, it isn't always the right call.

• Prototypes where what to build is itself undecided. Filling in the table is the waste. Build it, look at it, throw it away.
• Areas with genuinely many correct answers. Layout micro-adjustments, copy, recommendation ordering — pinning one expected value makes every legitimate change break a test. Use invariants (no overlap, count preserved) or visual regression instead.
• One-line fixes. You don't need a table to fix a typo.

Scaling the intensity to the risk is the realistic approach. Concentrate it where being wrong hurts — payments, permissions, data migrations — and go light everywhere else.

Wrapping up

Controlling an AI turned out not to be about writing better prompts. It was about where you put the authority to define correctness.

And one principle runs through all three.

When confidence runs out, don't pick the plausible answer — mark it unknown and stop.

Amusingly, the same idea shows up somewhere completely different. When restoring a highlight in a web document, one rule scores the candidate ranges and then refuses to attach anything if the gap between first and second place is small. Attaching to the wrong place is worse than not attaching at all.

One escalates a question to a human and the other gives up on a highlight, but the goal is identical: never produce a quietly wrong result.

Further reading

• Test oracles and where correctness comes from
• Measuring test strength with mutation testing
• Re-finding a text range in a document that changes