Verify BDPD¹ — Governance: cheap talk and sanctioning on a fragile commons¶

Purpose¶

Confirm that BDPD¹ (paper_01) is internally consistent, empirically backed, and bibliographically clean. The paper makes three load-bearing empirical claims:

The cheap-talk null. Pre-decisional communication does not improve preservation among LLM agents on the BDPD¹ substrate (\(d \approx 0.17\) at \(N = 5\), with the Cell C refinement showing that the lift attributed to cheap talk by an early pilot was driven by architecture, not by talking).
The graduated-ladder dominance. A \([1, 3, 10]\) graduated sanctioning ladder preserves the commons in 5/5 seeds versus the flat-amount control's 3/5 (D2 pilot).
The escalation-shape claim. A ladder-geometry sweep narrows the mechanism: the shape of the ladder (graduation), not the amount, is the load-bearing parameter. Soft [1,2,4] and hard [1,5,25] both work; flat [5,5,5] does not.

A pass means all three claims are recoverable from the pilot data on disk and the bibliography is clean.

Round-2 scope (2026-06-16+). Round 2 is a complete re-verification from zero, not a delta of round 1. Every claim, every citation, every number is re-checked. Round-1 verdicts are inputs only — they have no privileged status. Five audit dimensions are appended after the core Procedure: Numeric audit, Narrative-flow audit, Balance audit, Harmonization audit, Epistemic-honesty audit. The Quick reference table at the bottom lists every load-bearing number with its canonical anchor — consult it before re-deriving anything; it is the vaccine against the false-positive pattern from round 1 (DS pitch_01: 6/7 wrong findings from too-narrow grep, including a confusion of cell A/B/C/D semantic identity vs legacy pilot dir names that is especially relevant for this paper). Round-1 reports live at tmp/verification/<reviewer>_paper_01_report.md; round-2 reports land at tmp/verification_round2/<reviewer>_paper_01_report.md (do not overwrite).

Where to look¶

Paper sources¶

migration/docs/publications/paper_01/index.qmd
migration/docs/publications/paper_01/00-abstract.qmd
migration/docs/publications/paper_01/01-introduction.qmd
migration/docs/publications/paper_01/02-approach.qmd
migration/docs/publications/paper_01/03-experiments.qmd
migration/docs/publications/paper_01/04-discussion.qmd
migration/docs/publications/paper_01/glossary.qmd
migration/docs/publications/paper_01/series_coverage.qmd
migration/docs/publications/paper_01/_quarto.yml
migration/docs/publications/paper_01/references.qmd

Compiled output¶

migration/docs/publications/paper_01/_site/index.html and the matching PDF.

Bibliography¶

migration/docs/publications/paper_00/bdpd_full.bib — shared across the four papers.
migration/docs/library/<bibkey>.pdf and migration/docs/library/txts/<bibkey>.txt.

Key bibkey families to spot-check for this paper:

Cheap talk: farrell1996cheap, crawford1998survey, balliet2010communication, agranov2025communication, milinski2008collective.
Sanctioning: ostrom1990governing (graduated sanctions principle), fehr2000cooperation, yamagishi1986provision, klingeren2024graduated.
LLM-in-the-loop: park2023generative, rahwan2019machine, brookins2024playinggames, horton2026simulatedeconomicagents, gati2023usingLLMs.
Cognitive-science framing (added 2026-06-11): simon1955behavioral, gigerenzer2011heuristic anchor the rule-bound vs signal-conditioned architectural distinction.

Pilot data — the empirical record under the claims¶

migration/data/pilot/d1/ — initial cheap-talk pilot (n=3) and full sweep.
migration/data/pilot/d1_n5/ — N=5 cheap-talk pilot (gives the \(d = 0.17\) effect size).
migration/data/pilot/d1_cell_c/ — the LLM-no-talk Cell C refinement (4/5 preservation without talk). This is the data behind the finding that architecture, not talk, drives the lift.
migration/data/pilot/d2/ — graduated [1,3,10] vs constant [3,3,3] Mule pilot (n=1).
migration/data/pilot/d2_llm/, migration/data/pilot/d2_llm_n5/ — D2 with LLM agents.
migration/data/pilot/d2_ladder_sweep/ — [1,2,4] soft vs [1,5,25] hard vs [5,5,5] flat ladder geometries.
migration/data/pilot/c1/ — C-series pilot (sanctions firing dynamics).

Aggregate JSON, run logs (.log), and any plots produced under each directory are the artefacts the reviewer should consult.

Code¶

platform/pacts.js — pact registry (declared agreements).
platform/messages.js — cheap-talk channel and broadcast tools.
platform/reputation.js — per-player behaviour score.
platform/perturbation.js — sanctioning perturbations.
agents/governance.js — governance meta-agents.
agents/bdpd_sdk/ — the Python LLM agent SDK.
experiments/definitions/*.json — relevant sweep defs (B-, C-, D-series).

Procedure¶

Same five-step backbone as verify_paper_00.md:

Read PDF once, note every numerical claim.
Claim ↔ pilot data trace — for each claim, find the JSON/log under data/pilot/d* and confirm the number is recoverable. Pay particular attention to the Cell C refinement: an early pilot saw a 0/5 → 2/5 lift attributed to cheap talk, and the rewrite (paper_02 cherry-pick + paper_01 rework) re-attributed it to architecture. Verify the rewrite is consistent with d1_cell_c/.
Claim ↔ code trace — the cheap-talk channel and the sanctioning ladder are both implemented; confirm parameter values and broadcast semantics match the paper.
Bib audit — restrict to bibkeys cited in paper_01/*.qmd. Spot-check Ostrom, Fehr-Gächter, Yamagishi, Balliet, Farrell-Rabin against the actual PDFs.
Citation-in-context spot-check. Pick three citations on cheap-talk efficacy and three on sanctioning. For each:

a. Content accuracy. Grep migration/docs/library/txts/<bibkey>.txt for the most distinctive keyword in the paper's surrounding sentence (e.g. "Cohen's d", "graduated sanctions", "second-order public good"). Open the matching passage in the PDF. Confirm the source's actual claim matches what the paper attributes to it. Particularly: the literature reports \(d \approx 1.01\) for cheap talk in humans — the paper's framing must neither overstate nor understate that meta-analytic effect. b. Context appropriateness. Confirm the source is cited where its argument actually applies. A citation about human cheap talk used to support a claim about LLM cheap talk needs an explicit bridge in the paper — without that bridge, the citation is misplaced. c. Hedging match. Source meta-analyses are statistical; paper claims should preserve the confidence interval, not flatten it to a point estimate. d. Bib-key precision. As in verify_paper_00.md step 5.d.

Highest-priority spot-checks: balliet2010communication (cheap-talk meta-analysis), farrell1996cheap (theoretical anchor), ostrom1990governing (graduated sanctions principle), fehr2000cooperation (altruistic-punishment baseline), yamagishi1986provision (second-order problem). 6. Figure reproduce — pick one figure from the experiments chapter and regenerate via docs/experiments/reproduce.md. 7. Cross-paper consistency — BDPD¹ → BDPD²: confirm BDPD² re-uses BDPD¹'s cheap-talk null without distortion. 8. Render check — bash render_publications.sh --which paper_01. 9. Round-2 git-log waypoint. git log d0a26ff..HEAD -- docs/publications/paper_01/. Reading-priority hint, not scope limit.

Round-2 audit dimensions¶

The five dimensions below augment the core procedure. They are mandatory for round 2.

Numeric audit (round 2)¶

Purpose. Re-derive every literal number quoted by paper_01 from its canonical source, fresh eyes, no trust of the round-1 verdict. The 2×2 factorial cell-aliasing trap is unique to this paper: every number quoted as "cheap talk effect" risks a wrong-cell-comparison if the reviewer does not consult the Quick reference table first.

Procedure.

Numeric ledger of every literal in paper_01/*.qmd.
Source-of-truth precedence: regression baseline > paper qmd > pilot aggregate. For the 2×2 cells, the paper qmd is the source of truth for which cell-letter maps to which condition (built-in vs LLM × no-talk vs with-talk); the legacy pilot dir names (cell_B_on = LLM-with-talk, not "built-in with talk") are misleading and must be resolved through the Quick reference table.
Recompute the 33% / +17% wealth delta by hand: (cell_B_on.aggrWealth - cell_C_llm_notalk.aggrWealth) / cell_C_llm_notalk.aggrWealth = (119.464 - 177.278) / 177.278 = -0.326 ≈ -33%. Same for cooperators.
Cohen's d = 0.17 with pooled σ ≈ 26 must be recoverable from paper_01/04-discussion.qmd:30 (pooled σ value) + the cell-B-vs-cell-C commons stock means.

Acceptance. Same four-row table as the paper_00 numeric audit.

Narrative-flow audit (round 2)¶

Identical to paper_00's. One specific Cell-C lineage check: the rewrite that re-attributed the cooperation lift from cheap-talk to architecture must be narratively complete — no orphan paragraphs from the pre-rewrite framing.

Balance audit (round 2)¶

Identical to paper_00's. Paper_01 specific: confirm the D1 (architecture decomposition) section is the narrative center; the D2/D3 (sanctioning ladder) sections complement it but should not dominate.

Harmonization audit (round 2)¶

Identical to paper_00's. paper_01 specific:

paper_01/glossary.qmd should align with docs/publications/glossary.qmd and with paper_00's terminology.
series_coverage.qmd declares which sanctioning ladders were tested at which scales — confirm the matrix is faithful to the pilot data on disk.
Cross-paper: paper_02's references to paper_01's cheap-talk null must match how paper_01 characterises it.

Epistemic-honesty audit (round 2)¶

Identical to paper_00's. paper_01 specific:

The cheap-talk null must be presented as such; resist any phrasing that softens it (e.g. "weak effect" instead of "no detectable effect at this sample size").
The "humans show \(d \approx 1.01\), our LLMs show \(d \approx 0.17\)" framing must not overclaim either direction — the LLM result is a single-condition \(N=5\) pilot; human meta-analysis is 45 studies.

Acceptance criteria (composite)¶

Check	Pass	Partial	Fail
Claim ↔ pilot data	every claim mapped to a recoverable JSON/log	1–2 unmapped	majority unmapped
Claim ↔ code	substrate matches paper's description	minor parameter drift	code disagrees with paper
Bib audit	every cited bibkey has a PDF	1–2 missing	≥3 missing
Citation accuracy spot-check (3 talk + 3 sanction)	all faithful	one loose	misrepresentation
Figure reproduce	matches paper	minor stylistic delta	numerical difference
Cross-paper consistency	BDPD² cites BDPD¹ faithfully	minor framing shift	misrepresentation
Render	clean HTML + PDF	minor glitch	broken
Numeric audit	every literal traced (Quick reference + pilot recompute)	1–2 untraced	systemic
Narrative-flow + Cell C lineage	rewrite complete	residual stale framing	systemic
Balance audit	D1 center, D2/D3 complement	one outlier	systemic
Harmonization audit	glossary + series_coverage aligned	1 drift	systemic
Epistemic-honesty audit	null framed honestly, no overclaim	1 borderline	systemic

Common pitfalls¶

Unicode + pipe-table rules as in paper_00.
d2_llm_run.log and d2_llm_n5_run.log at top of data/pilot/ are loose log files, not directories.
The cheap-talk channel had a --no-talk flag added during the v1.1 development that produced Cell C. Confirm the flag is documented in docs/platform/cheap-talk.md.
The lie_score metric is dead as of the D1 pilot findings (zero across all conditions). silent_defection is the live indicator.
Cliff vs collapse — terminology discipline. "Cliff" is reserved for Bardi/Seneca asymmetric collapse (BDPD³). paper_01 builds on BDPD⁰'s collapse finding; do not flag "cliff" if it appears in a quote from BDPD⁰'s cliff edge terminology, but do flag any newly-introduced "cliff" used as a synonym for collapse.
Round-1 false positives — do not re-trigger. The 2×2 factorial cell A/B/C/D aliasing is the highest-risk trap for paper_01 reviewers. The legacy pilot dir naming maps differently from the paper's semantic naming:
data/pilot/d1_n5/cell_A_off = built-in no-talk → semantic cell A.
data/pilot/d1_n5/cell_B_on = LLM with-talk → semantic cell D, not cell B.
data/pilot/d1_cell_c/cell_C_llm_notalk = LLM no-talk → semantic cell C.
Semantic cell B = built-in + talk = OUT OF SCOPE, never run. When the paper quotes "-33% aggressor / +17% cooperator from enabling communication", that compares semantic C (LLM no-talk, in d1_cell_c/) vs semantic D (LLM with-talk, in d1_n5/cell_B_on). Comparing cell_A_off vs cell_B_on (built-in vs LLM) gives -14% / +3% — a wrong-comparison artefact. Consult the Quick reference table before recomputing.
migration/ is the working tree, not the repo root. Path checks fail silently if cwd is bdpd/.

Quick reference — facts you should not re-derive¶

Claim	Value	Canonical anchor
D1 cell A (built-in no-talk) preservation	0/5 reached \(T=30\)	`01-introduction.qmd:98-106`, `data/pilot/d1_n5/cell_A_off` (aggrWealth=139.32 deterministic, sd=0)
D1 cell C (LLM no-talk) preservation	4/5 reached \(T=30\)	`00-abstract.qmd:14-25`, `data/pilot/d1_cell_c/aggregate.json` (aggrWealth=177.278, sd=20.073)
D1 cell D (LLM with-talk; legacy name `cell_B_on`) preservation	2/5 reached \(T=30\)	`00-abstract.qmd:14-25`, `data/pilot/d1_n5/cell_B_on` (aggrWealth=119.464, sd=19.214)
Aggressor wealth delta (talk on)	\(-33\%\) = \((119.464 - 177.278) / 177.278 = -0.326\)	`00-abstract.qmd:14-25` (cell C → cell D, not A → D)
Cooperator wealth delta (talk on)	\(+17\%\) = \((54.441 - 46.608) / 46.608 = +0.168\)	`00-abstract.qmd:14-25`
Cohen's d (cell B-vs-C, commons stock)	\(d = 0.17\)	`00-abstract.qmd:18`, `01-introduction.qmd:107`, `03-experiments.qmd:231`, `04-discussion.qmd:30,108,179`
Pooled σ (D1 cells)	\(\sigma \approx 26\)	`04-discussion.qmd:30`
D2 graduated \([1,3,10]\) preservation	\(5/5\)	`00-abstract.qmd:27-30`, `data/pilot/d2_llm_n5/cell_A_graduated_llm`
D2 flat-3 control preservation	\(3/5\)	`00-abstract.qmd:27-30`, `data/pilot/d2_llm_n5/cell_B_constant_llm`
D3 ladder sweep: flat-5	\(1/5\)	`00-abstract.qmd:31-33`, `03-experiments.qmd:404-413`, `data/pilot/d2_ladder_sweep/cell_C`
D3 ladder sweep: soft \([1,2,4]\)	\(4/5\)	same, `cell_A`
D3 ladder sweep: canonical \([1,3,10]\)	\(5/5\)	same
D3 ladder sweep: hard \([1,5,25]\)	\(5/5\)	same, `cell_B`
C1 sanction firing	1:1 mechanism, \(6 \pm 3.5\) violations → \(6 \pm 3.5\) sanctions over 5 seeds	`03-experiments.qmd:75-99`, `data/pilot/c1/aggregate.json`
Cheap-talk lit reference effect size (humans)	\(d \approx 1.01\) across 45 studies	`01-introduction.qmd:59`, `library/balliet2010communication.pdf`

Output¶

A report at tmp/verification_round2/<reviewer>_paper_01_report.md with:

Header (reviewer, mode, commit, date).
2×2 cell ledger — explicit table mapping semantic A/B/C/D ↔ legacy pilot dir name ↔ pass/partial/fail.
Verified-claim ledger.
Bib audit table.
Numeric ledger (Quick reference rows matched or recomputed).
Round-2 audit tables — one per dimension (Numeric / Narrative-flow + Cell C lineage / Balance / Harmonization / Epistemic-honesty).
Findings list with severity.
Render notes.

Reviewer prompt (copy-paste)¶

You are an independent reviewer verifying BDPD¹ — Governance: cheap talk and sanctioning on a fragile commons, completely and from zero — this is round 2, no round-1 verdict is inherited. The repository is at migration/ on the round-2 working branch (or tag v1.1.2). Follow the playbook at migration/docs/verification/verify_paper_01.md from top to bottom: core Procedure (steps 1–9) plus the five Round-2 audit dimensions (Numeric / Narrative-flow + Cell C lineage / Balance / Harmonization / Epistemic-honesty). Before starting the Numeric audit, read the Quick reference table at the bottom of the playbook AND the Common-pitfalls entry on cell A/B/C/D aliasing — the 2×2 factorial cell-name confusion is the highest-risk trap for this paper and produced the round-1 false positives in DS's pitch_01 audit. The paper's load-bearing claims are listed in the Purpose section. Confirm: (a) the Cell C refinement narrative (architecture-not-talk attribution) is complete and consistent with d1_cell_c/; (b) the graduated [1,3,10] ladder preservation is recoverable; (c) the ladder-geometry sweep shows the shape effect; (d) the bibliography is clean; (e) the five audit dimensions pass. Produce the report at tmp/verification_round2/<reviewer>_paper_01_report.md. Time budget: roughly one and a half working days.