RelayOps
Case studies

Three runs you can reproduce

One for each outcome. Each comes with the signed Passport, the public key to check it, and the exact commands to get the same verdict.

VERIFIED

A real fix, independently re-run

A refund bug fix with a new test. RelayOps re-ran it in a clean sandbox: the new test fails on the base branch and passes on the change, so the fix fixes the bug it claims to. The tests also caught every bug RelayOps planted in the changed lines.

Checks
tests 4/4 · reproduce: fails on base, passes on head · test strength: strong (1 of 1 planted bugs caught) · test integrity: no weakening · secrets, dependencies, code scan: 0 findings
Passport
01M3X324D464V00VR0CE42BZYN · verify it in your browser
Limits
A small planted repository: it shows the mechanism, not performance on a large codebase.
FAILED · test integrity

Tests weakened to get green

A "fix" that breaks legacy charges, then deletes one assertion and loosens another so the tests pass. The commit carries an AI co-author trailer. The change's own tests pass, so CI is green. RelayOps re-ran the base branch's test against the new code: it fails. The Passport names the exact line.

Checks
tests 4/4 (green) · test integrity: fail, assertion removed at test/refunds.test.js:7 plus one loosened, confirmed by execution · test strength: weak (2 of 3 planted bugs caught)
Passport
01M3X32GRHNX67MKG2W2TH9WKY
Limits
A planted repository. In the 15 real merged pull requests measured so far, none weakened an assertion to get green.
INCOMPLETE

A real agent pull request that skips test suites

e2b-dev/E2B #1863 made 13 JavaScript test files and one Python suite skip unless an environment variable is set. The maintainers merged it on purpose (it tests a deprecated API). RelayOps listed all three skip mechanisms with file and line, and refused to call the change verified or failed: the suites need an API key the sandbox doesn't have, and the Passport names it.

Checks
test integrity: 3 skip signals, can't be confirmed without the key · tests: 9 passed, 4 failed with authentication errors, the same 4 failing on the base branch, so recorded as "needs secret" and not as the change failing
Passport
01M3P5KYZZWS0P76WX88204E31
Limits
An earlier engine version found only 1 of the 3 skip mechanisms; the current one finds all 3. RelayOps doesn't inject secrets into tests yet (planned).

Re-run them yourself

relayops verify fixtures/.generated/node-refunds --base main --head clean-fix        # case 1
relayops verify fixtures/.generated/node-refunds --base main --head agent-weakened   # case 2
relayops-passport verify passport.dsse.json --key passport-keys.json --allow-dev-key

The full method for the 15-PR measurement (which changes, what counted as correct, and what it doesn't show) is shared with design partners on request. All three Passports here are signed with development keys.