Before you code, build the harness

Claude Code reads your CLAUDE.md at the start of every session. That does not mean it is bound by it. Anthropic's own documentation says Claude treats the file as context, not enforced configuration. Nothing in a CLAUDE.md stops an agent from calling a task finished on the strength of a test run from 3 edits ago, and a longer CLAUDE.md does not change that. Something has to check the work, and it has to be something that can fail.

So I built claude-code-harness, a small starter kit for using Claude Code on a project that already has tests. It is everything around the agent rather than the agent itself: the instructions it reads, the workflow it follows, the checks that run, and the feedback that reaches it. Claude Code's own runtime is untouched. Every piece is a file Claude Code already knows how to read.

Context, tools, checks

The kit has 3 parts, and only the third one enforces anything.

• Context: a CLAUDE.md template for the stable facts (purpose, constraints, commands, definition of done), a task brief for 1 task's acceptance criteria, and a handoff note for the evidence at the end. Stable rules and per-task criteria live in different files so neither buries the other.
• Tools: 1 skill, verified-change. Understand the code, plan when the task warrants it, make a bounded change, run the agreed checks, read the diff, report the evidence and what is still unverified.
• Checks: verify.py, which runs the commands the project already uses and decides whether a change is done, plus an optional Stop hook that runs it when Claude tries to finish.

Two details came straight from reading the documentation before writing any code. The skill is not called verify, because a project skill with that name replaces Claude Code's built-in /verify command and changes its commit instructions. And the CLAUDE.md template ships as CLAUDE.template.md, because Claude Code loads any file named CLAUDE.md in a subfolder as live instructions the moment it reads a file there. A template full of placeholders would have become part of the agent's context.

A pass has to belong to a source state

verify.py runs each configured command as an argument list, without a shell, with a time limit per check and for the whole run. On a timeout it kills the process tree, not just the parent. It keeps the real exit code and separates 5 outcomes: passed, failed, timeout, skipped and not configured. A required check with no command fails. It is never quietly treated as a pass.

The decision the whole tool rests on: every result is tied to a SHA-256 fingerprint of the selected source files and of the checks configuration. Change a source file and the old pass stops counting.

$ python .claude/harness/verify.py statusevidence  STALEreport    .claude/harness/evidence/latest.json (PASSED, finished 2026-10-10T19:10:14Z)recorded  74eb229d6ac5current   2e1c003125f9  modified  invoice_totals/totals.pyThe recorded result does not apply to the current source. Run verify.py again.

An agent saying the tests pass is a claim about some earlier version of the code. This makes the claim checkable. The evidence names the source it ran on, and status exits 3 the moment that source moves. Editing a file outside the selected source, such as CLAUDE.md itself, leaves the evidence fresh, because the include patterns decide what counts.

A hook that cannot trap the agent

The optional Stop hook runs when Claude finishes a turn. If the source is unchanged since the last report it reuses that report. Otherwise it runs the checks. When a required check fails it returns a block decision with the failure output, so Claude keeps working with the error in front of it.

The hard part is making sure it cannot loop. Claude Code caps Stop hook continuations at 8 in a row, but that count resets every time Claude calls a tool, and an agent fixing code calls tools constantly. So the hook carries its own limits: at most 2 blocks in a row, never a second block for the same unchanged source, and it never exits with code 2, so a crash or malformed input becomes a visible error instead of another forced turn. A missing or invalid checks file produces a message, not a block. Claude should not be pushed into writing the commands it is about to be judged by.

Instructions guide the agent. Checks decide.

What happened in a real session

Fixture tests show the hook follows the documented contract. They do not show that Claude Code acts on it. So I ran it for real: Claude Code 2.1.296, a temporary copy of the example project, the hook enabled only in that copy, and my personal settings and connectors switched off for the run.

The example is a small invoice module with a real bug. VAT is rounded with Python's round(), which rounds half to even on a Decimal, so 1.50 at 19 percent comes out as 0.28 instead of 0.29. In the first scenario Claude worked through the task brief with the skill. The checks failed, it fixed the rounding, the checks passed, it read the diff, confirmed the evidence was fresh and wrote the handoff note: 14 turns in 53 seconds. The hook ran once at the end and let it stop, because the passing evidence matched the source.

The second scenario is the one that matters. I started from the fixed code and asked Claude to switch the rounding to half-even, make only that edit, and run nothing. It made the edit and tried to stop. The hook ran the checks, 2 tests failed, and it blocked. The failure output reached Claude as Stop hook feedback. Claude did not touch the tests. It stopped and wrote:

The edit you asked for causes both failures. The tests check the half-up rounding that CLAUDE.md requires, and half-even rounds those half-cent amounts down instead.

It then laid out the 2 options and asked which one I wanted. On its next stop the source had not changed since it was told, so the hook let it finish and reported why instead of blocking again. 1 block, 5 turns, 22 seconds. Both session records are in the repository, with local paths and account details stripped.

Checking the checks

A check that has only ever been seen to pass has not been shown to catch anything. So the repository includes a script that breaks the code on purpose, 1 defect at a time in a fresh copy, and requires the matching test to fail: a timeout that leaves the process tree alive, a fingerprint that ignores file content, a hook that ignores its retry limit, an installer that follows a link out of the target folder.

$ python scripts/mutation_check.pyApplying 26 deliberate defect(s), each in a fresh temporary copy....26 of 26 defects caught by the named test.

It earned its place on the first run. One of my own tests was meant to prove the hook reads a UTF-8 payload correctly on Windows, but the JSON it sent was ASCII-escaped, so it never exercised the decoding at all. Breaking the code exposed it. I also had to retract a claim of my own: I had written that Claude Code refuses to start inside another Claude Code session. I had not tested it, and it was wrong. The suite is 78 tests on Python 3.11 and 3.13, and GitHub Actions runs it on Ubuntu, macOS and Windows.

What it does not do

• It is not a sandbox. Listing a command in checks.toml authorises it to run with your permissions.
• It is not a security boundary. A local hook can be edited or switched off by anyone who can write to the repository, Claude included. For things that must never happen, use Claude Code's permission rules and its sandbox.
• It does not try to spot dangerous shell commands with a pattern list. That kind of filter looks like protection and is not.
• It makes no claim that you or Claude get faster or better. 2 recorded sessions are examples, not a measurement.

It is also not llm-eval-harness, my earlier repository, which scores language model answers against expected results. This one does not score a model at all. It runs your project's own checks against an agent's changes and records whether they passed on the code as it is now. It is an independent project, not made or endorsed by Anthropic.

Try it

git clone https://github.com/agentdynarq/claude-code-harnesscd claude-code-harnesspython -m unittest discover -s testspython scripts/demo.pypython install.py install path/to/your-project --dry-run

The demo copies the example to a temporary folder, runs the whole walkthrough for real and checks 14 expectations. The installer only creates files, refuses on any conflict, and never touches your CLAUDE.md, settings or hooks. Standard library Python 3.11 or newer, no dependencies, MIT.