Shipmoor

Claim Check overview

Shipmoor Team
•
July 13, 2026
•
7 min read

Claim Check is a verification tool that runs after your coding agent finishes and checks whether the change did what the task asked. It doesn’t review style or catch structural defects; that’s the job of the structural scan. Claim Check answers a narrower, harder question: did this diff actually do the thing it claims to have done?

Like the rest of Shipmoor, it runs locally. No source leaves your machine, and Shipmoor hosts no model of its own. The result is a reproducible verdict: the same frozen acceptance set and the same fixed evidence set produce the same, byte-identical verdict, and you can replay it later, offline, from what it wrote to disk.

The problem it solves

A coding agent can produce a diff that lints clean, type-checks, and passes the tests it wrote, and still do the wrong thing: wire the wrong event, edit a sibling service, or “handle” an error by quietly swallowing it. None of that shows up as a lint error. Reviewing agent output at volume is exhausting, so a confident-looking diff gets rubber-stamped, and the gap between what the agent said it would do and what the diff actually does is where these changes hide.

That’s the gap Claim Check catches, before a human reviews the diff. Where it has a real check for something, it runs it and reports the outcome. Where it doesn’t, it says so, “couldn’t check”, instead of implying a pass.

The acceptance set

Claim Check evaluates a change against an acceptance set: a list of atomic, binary obligations written in plain language, one line each, for example “a handler is bound to the payment_intent.payment_failed event.” Each obligation is an acceptance criterion, often shortened to AC or called an acceptance item. Once resolved, the set is frozen: its content is hashed, and the tool re-checks that hash on every run, so nobody can quietly loosen the goalposts after a verdict has already been produced.

You don’t have to hand-write the set every time. If none is committed, Claim Check derives one from your stated intent (--intent-ticket or --intent-prompt; see Providing intent) and writes it as an unapproved draft to .shipmoor/acceptance.yaml.draft. A draft is a proposal, not an agreement, and it never earns a full pass: a run against a draft reads INCONCLUSIVE instead of READY, no matter how clean the diff is, until a person reviews it and commits it as .shipmoor/acceptance.yaml. Committing it is what turns a guess about intent into a claim Shipmoor will actually hold the change to.

The floor and the judge

Claim Check runs in one of two modes, and both share the same chassis.

The deterministic floor runs first, always. It binds each acceptance criterion to a check, a command’s exit code, a git ref, a bound test, and it never guesses. Its own internal signal is strictly binary: READY or BLOCKED. Anything it can’t check mechanically is left open, never marked passed by default.

shipmoor claim-check .                 # floor-only, if no --agent is configured
shipmoor claim-check . --agent claude  # full mode: the floor, then the judge

Floor-only mode (--floor-only, or simply no --agent) stops there: fast, no model, no secret. Full mode (--agent "<byo-agent-cmd>") adds an intent fidelity judge on top of whatever the floor left open. claude is the only bare preset with a built-in headless invocation; point --agent at a raw command to run any other coding-agent CLI as a subprocess. The judge can BLOCK, citing a specific divergence, or ABSTAIN, routing the question to a human. It can never PASS an obligation. A green verdict always rests on the deterministic floor; it is never the judge’s word alone. Full mode needs the claim_check entitlement. See BYO-Judge and Plans & tiers for what each tier unlocks.

Binder, not oracle

Claim Check doesn’t know whether your code is correct. Your own toolchain, your compiler, your test runner, your type checker, is the oracle. Shipmoor’s job is narrower and more mechanical: it binds an obligation to a check, runs that check, and records what happened. When there’s no check to bind to yet, it says so instead of guessing. That’s also why the judge can never PASS an obligation: passing would mean vouching for correctness, and the real evidence lives in the floor’s deterministic checks, never in a model’s opinion of them.

When Shipmoor can’t decide: the fix packet

Some obligations are judgment calls a machine can’t fully close on its own, at least not yet. Claim Check doesn’t just leave those as a shrug. New in 0.8.0, every run that isn’t fully READY emits a fix packet alongside the verdict: a structured list of exactly what’s wrong and what to do about it.

The fix packet has three parts:

  • fixes: one entry per requirement that actually failed, either a deterministic floor deny or a judge block the floor accepted. Each entry names the requirement, the acceptance criterion it belongs to, the reason, and a concrete evidence pointer, an attestation digest, a file:line, or a test id. Never a bare assertion.
  • route_to_human: the acceptance items that came back abstained or otherwise undecided and need a person’s judgment. This is the INCONCLUSIVE list, made actionable.
  • author_checks: the authoring-loop payload, and the part worth understanding well.

For every required, judgment-class obligation that isn’t bound to a real check yet, author_checks names the exact obligation statement and hands you an exact tag string. You, or your coding agent, write a new test whose id or filename carries that tag. Claim Check matches the test to the obligation by an exact substring join, nothing fuzzy, no model in the loop for the match itself. Then it asks for proof the test is real: the check must fail at the base commit and pass at the head commit. A test that was already green before your change proves nothing, so Claim Check won’t accept it as evidence.

Once a check like that is bound and passing, its obligation graduates for good: what used to be something a judge had to guess about is now something the deterministic floor decides on its own, every run, for free. Put plainly: when Shipmoor can’t decide something on its own, it tells you exactly what test to write and what to name it, then it checks that test itself before trusting it, rather than trusting your word that you wrote the right thing.

It’s worth being precise about what this is not. Claim Check confirms the test exists, is tagged correctly, and genuinely flips from failing to passing across your change. It does not read the test’s assertions to judge whether they truly exercise the claimed obligation. The tag match and the fail-to-pass transition are the evidence, not a content review of what the test asserts.

See Reading the verdict for a full worked example of a fix packet, field by field.

One shared evidence surface

Claim Check doesn’t work from the diff in isolation. A run evaluates several evidence producers concurrently over the same change: the scan, review if you’re entitled to it, your build, your test suite, test integrity checks, artifact validation, every acceptance item with a bound check, and, in full mode, the judge. For entitled callers, it also folds in Test Evidence: whether your tests actually passed, whether the test surface shrank, whether a kept test was quietly gutted. Test Evidence runs as one more producer alongside the others and feeds straight into the same verdict. See Attestation artifacts for how the outcomes are recorded, and Test Evidence overview for the full story on that signal.

Next

Last updated on July 13, 2026

Was this article helpful?

Your response is saved on this device.

Related Articles