“…fetched through safeFetch behind an always-on assertUrlIsPublic SSRF check…”
Claim Check splits a pull request’s description and review thread into every claim a reader could falsify, then gives each claim to its own IBM Bob subagent, all running in parallel. Each one comes back with a verdict and the line of code that settles it. On this PR, one of 29 came back false — and it was the one the reviewers had signed off on.
The signal path does call assertUrlIsPublic. The Test button in the credentials dialog did not: it called the raw fetch on a URL the operator types, and reported the HTTP status back — a probe of any host the operator can name, including private ranges.
Every outbound request passes through assertUrlIsPublic as an always-on SSRF check.
1025 const rawUrl = 1026 credentials.PREDGE_SIGNAL_URL?.trim() || DEFAULT_PREDGE_SIGNAL_URL; 1027 const baseUrl = stripTrailingSlashes(rawUrl); 1028 1029 // Read-only: the published keyset confirms the signal service is reachable 1030 // and is serving Predge signing keys. 1031 const response = await fetch(`${baseUrl}/.well-known/predge-keys.json`, {1032 method: "GET", 1033 headers: { Accept: "application/json" }, 1034 });
No assertUrlIsPublic, no safeFetch. The URL comes straight from operator credentials. We wrote this code, fixed it the same night with six tests, and disclosed it in the PR thread ourselves.
Open any row for the author’s original words, the subagent’s reasoning, and — where we re-ran it live through Bob Shell — the live result. Evidence links open the exact line of the captured diff.
The maintainer’s final review names four conclusions under “Does it match the description”. Those four cover 14 of the 29 claims, and the tool agrees with him on all 14. He did not address the other 15 one by one — and the failing claim is among them.
To save budget, one build of the live executor ran each subagent on the diff alone. It returned holds for c02: it found the guarded path, and stopped.
“Every request is guarded” is a universal claim. One example cannot confirm it; only a counterexample can refute it. Universal claims now get a different instruction: search every call site, list what was checked, and answer cannot tell if coverage isn’t proven. With that rule the live run finds the bug again.
The worst defect we know of in this codebase is a mismatch between two canonicalisation functions: the signer drops undefined keys, the verifier does not. Identical bytes today; incompatible the day a field becomes optional.
No sentence in the PR says anything about it, so no claim was extracted and no subagent looked. That is a boundary of the method, not a bug in Bob — and it is the same reason four rounds of review walked past it.