addyosmani

adverse

Multi-agent adversarial code review for any coding agent

by addyosmani

JavaScript

JavaScript100%

Commit activity

Last 12 weeks · 0 commits

Full graph →

Community health

2 of 6 standards met

Community profile →
42
✓README✓License○Contributing○Code of Conduct○Issue Template○PR Template

Recent PRs & issues

Quiet · 1 discussion · Last activity 4mo ago
See all on GitHub →
kfox

Withdrawn. This was opened against the wrong repository by mistake and closed 16 seconds later; it was never intended for upstream and was never meant to ask for anyone's time. The changes belong to a downstream experimental fork and are not proposed here. Nothing in this PR needs review, and there is no request attached to it. Apologies for the noise.

kfox · 2mo ago
kfox

A four-lane adverse review of everything on since e40da19 (#21) found this. CRITICAL, consensus — reported by the Adversary, validated in round 2, with a second site found by the round-2 Auditor and a third by the Steward. The guard was written once and not applied at the sites that needed it : comes straight out of a reviewer-written round-2 payload with no registry check — unlike the sibling bridge , which added exactly this check in this same change, with the rationale spelled out in its own header: the persona string keys the output filename … a re-cased or invented name would mint a phantom reviewer So the reasoning was written down and then not applied to the bridge sitting next to it. Three sites, one missing guard 1. * — no roster check at all. An invented or re-cased persona mints a phantom reviewer file; a repeated one silently overwrites the previous lane's repaired payload before reads the glob. 2. — has the roster check but not the destination check. Two payloads for one persona collapse onto one file. (Found by the round-2 Auditor and independently by the Steward: "verify.mjs shares repair.mjs's silent-overwrite hole".) 3. — filters and throws the rest of the plan away, so hands combine the full lane roster and combine only mines it for split lanes. Its only persona check is membership in — so a payload from a lane the plan marked is accepted as a reviewer. SKILL.md Phase 5 invokes it as a glob over , a directory several subagents can write to. already treats a duplicate persona as an error and exits 1. That is the right posture; the other two bridges do not share it. Why it matters here specifically Consensus is this tool's entire product, and the synthesizer counts distinct personas. A phantom or duplicated persona file is the cheapest possible way to counterfeit the one signal the design trusts most — SKILL.md says so in as many words ("two independent-looking reports of one issue is exactly what synthesis reads as cross-validated consensus"). The inputs here are LLM output by design, so this does not require a malicious actor, only a model that re-cases its own persona name. Related, same root: centralized the wrong half was added in this range to give every bridge one /usage helper "so exit 2 means the same thing everywhere". It took and left the persona guards duplicated in three bridges — which is how the three sites above drifted in the first place. 's likewise keeps its own copy of the plan-shape guard already has. Suggested fix Put both halves of the guard in and have , , and call the one implementation: Specifically, before any write: Reject a payload whose is not in . Derive the expected persona from the input filename, the way 's does, and require to agree. Refuse to write a this process has already written, or that already exists in this run — two payloads for one persona is a stale file or a spoof, never a legitimate state. And have return the full roster ( for every lane), so combine can reject a payload from a lane the plan did not run and warn when a lane the plan did run produced nothing. Pass to the round-2 combine too. One shared implementation, not four corrected copies — four copies is how these drifted. Adverse review, base e40da19, 6 round-1 agents + 3 round-2 cross-reviewers.

kfox · 2mo ago
kfox

A four-lane adverse review of everything on since e40da19 (#21) found this. CRITICAL, consensus — reported independently by the Auditor and the Adversary, reproduced end-to-end, and all three round-2 lanes named it as the single largest shared root cause in the run. One cause, five symptoms A reviewer payload is LLM output — the explicitly untrusted input of this tool. () checks , , and . It never type-checks , or 's type. The triage bridge then stores them verbatim (): Every downstream sink assumes a type nothing established, and each one fails open or crashes: Plus the same shape one field over: a non-string passes and crashes with a at . Reproductions actually run The Adversary ran the real bridge in its own checkout: a payload with in one lane and any other lane whose mentions that finding's file hung indefinitely. The round-2 Adversary independently reproduced the crash variant: . For #3 the Adversary ran it and got back , , , plus the sympathetic "legitimate for a latent defect" note — i.e. the tool vouching for an anchor it never checked. Note that #1 does not need an attacker. is an entirely plausible slip for a model asked to cite a line. Why this is one issue and not five All three round-2 lanes converged on this independently. The Auditor: "one ingest coercion plus fail-closed sinks closes all five." The Steward: "one coercion at the bridge's ingest block plus a fail-closed guard in checkClaim closes all five." Fixing them one sink at a time is exactly the "close the instance, not the class" failure — and #1 and #2 are literally the same line reported twice, by two different lanes. Suggested fix Two layers, because either alone leaves the class open: 1. Coerce at ingest — in the block of : and require / in . 2. Make each sink fail closed independently, so a future caller that skips ingest cannot reopen it: : before the test, — the same predicate already uses, so the two cannot drift apart again. : stop building a regex from data at all. Compare against a tokenized scan of ( matches, membership-tested against ) — O(len), and it cannot backtrack. : return for a non-string argument. : . Where one failure mode is silent and the other noisy, choose noisy: #3 and #4 are the dangerous ones, because they do not crash — they mint a verified-looking anchor out of a value nobody checked. Adverse review, base e40da19, 6 round-1 agents + 3 round-2 cross-reviewers.*

kfox · 2mo ago
Structured data for AI agents

Repository: addyosmani/adverse. Description: Multi-agent adversarial code review for any coding agent Stars: 68, Forks: 9. Primary language: JavaScript. Languages: JavaScript (100%). License: MIT. Open PRs: 0, open issues: 1. Last activity: 4mo ago. Community health: 42%. Top contributors: addyosmani, hashahid.