Skip to content

Verify BDPD⁰ — The foundational paper

Purpose

Confirm that BDPD⁰ (paper_00) is internally consistent, empirically backed by the pilot data the paper claims to draw from, and bibliographically clean (every citation resolves to a PDF in the library). The paper makes three load-bearing empirical claims:

  1. The single-aggressor collapse. A single aggressive agent is sufficient to drive collective survival to zero across the realistic-regeneration band (\(r \lesssim 0.9\)), independent of pool size, regen rate, initial endowments, or turn order. (The localised threshold itself is called the cliff edge at \(i \approx 0.05\) — the point where the collapse becomes inevitable — but the finding is the collapse, not a Seneca cliff in Bardi's asymmetric sense.)
  2. The reactive penalty. Locally reactive strategies (extract less when stock falls) under-perform unconditional cooperation collectively.
  3. The tragedy of the compensator. A defector who exploits a cooperator's slack profits by ~\(+27 \%\) without triggering collapse — the cooperator's virtue subsidises the defector's profit.

A pass means all three claims have a recoverable evidence chain (paper text ↔ pilot data ↔ code) and every citation is backed by a PDF on disk.

Round-2 scope (2026-06-16+). Round 2 is a complete re-verification from zero, not a delta of round 1. Every claim, every citation, every number is re-checked. Round-1 verdicts are inputs only — they have no privileged status. Five audit dimensions are appended after the core Procedure: Numeric audit, Narrative-flow audit, Balance audit, Harmonization audit, Epistemic-honesty audit. The Quick reference table at the bottom lists every load-bearing number with its canonical anchor — consult it before re-deriving anything; it is the vaccine against the kind of false-positive that occurred in the round-1 DS pitch_01 audit (six wrong findings from too-narrow grep). Round-1 reports live at tmp/verification/<reviewer>_paper_00_report.md; round-2 reports land at tmp/verification_round2/<reviewer>_paper_00_report.md (do not overwrite).

Where to look

Paper sources

  • migration/docs/publications/paper_00/index.qmd — book preface
  • migration/docs/publications/paper_00/00-abstract.qmd
  • migration/docs/publications/paper_00/01-introduction.qmd
  • migration/docs/publications/paper_00/02-approach.qmd
  • migration/docs/publications/paper_00/03-experiments.qmd
  • migration/docs/publications/paper_00/04-discussion.qmd
  • migration/docs/publications/paper_00/A-01-platform.qmd — platform appendix
  • migration/docs/publications/paper_00/A-02-humbaba.qmd — card game appendix
  • migration/docs/publications/paper_00/A-B-supplement.qmd — statistical supplement
  • migration/docs/publications/paper_00/_quarto.yml — render config (PDF metadata, biblio)
  • migration/docs/publications/paper_00/references.qmd — bibliography placeholder

Compiled output

  • migration/docs/publications/paper_00/_site/index.html and the matching PDF — the form the reviewer should read for any narrative claim.
  • migration/docs/publications/paper_00/_site/*.html — per-chapter HTML if cross-checking individual sections.

Bibliography

  • migration/docs/publications/paper_00/bdpd_full.bib — the bib file. paper_01–03 also reference this same file, so every entry should be checked here.
  • migration/docs/library/<bibkey>.pdf — primary source PDFs (134 currently). Every bibkey cited in the paper must have a matching .pdf here.
  • migration/docs/library/txts/<bibkey>.txt — extracted plain text from each PDF (153 currently). Useful for grepping for the exact phrase the paper attributes to a source.

Pilot data — the empirical record under the claims

  • migration/data/pilot/v10_d1/ through migration/data/pilot/v10_d5/ — the D-series pilots on the v1.0 substrate (single-aggressor collapse, reactive penalty, ladder).
  • migration/data/pilot/v10_seed_sweep/ — seed-determinism evidence; relevant for claims of robustness across seeds.
  • migration/data/pilot/c1/ — C-series (cheap-talk null and friends; mostly used by paper_01 but BDPD⁰ refers to the cheap-talk baseline).

Code — the engine behind the experiments

  • engine/logistic.js, engine/commons.js, engine/metrics.js — the substrate.
  • agents/built-in.js — the five built-in strategies (Aggressive, Conservative, Adaptive, RCP, Random).
  • experiments/definitions/*.json — the sweep definitions. P1 (aggressive_fraction), P10 (perturbation_regen_shock), P11 (mule_strategy_override) and the adaptive_effectiveness / observability_noise sweeps are the canonical ones for BDPD⁰.
  • tests/regression/baseline/{aggressive_fraction,adaptive_effectiveness,perturbation_regen_shock,mule_strategy_override}.json — the numbers the paper is anchored to.

Procedure

  1. Read the paper PDF cover-to-cover once (~90 minutes) before opening any code. Take notes on every numerical claim — the specific number, the section, and the figure or table it lives in. This list becomes the spine of the verification.

  2. Claim ↔ pilot data trace. For each empirical claim in your notes, locate the pilot data directory and find the JSON / log / plot that produces the number. Open it, recompute the headline statistic by hand or with a quick jq query, and confirm the paper's number is recoverable to the stated precision.

  3. Claim ↔ code trace. For each engine-level claim (e.g. "the substrate is a single-stock logistic with Forest Die at 5%"), find the constant or function in engine/. Note any divergence — paper says one thing, code does another.

  4. Bib audit. Grep bdpd_full.bib for every @<type>{<bibkey>, …}. For each bibkey actually cited in paper_00/*.qmd (use grep -h "@<bibkey>" paper_00/*.qmd to filter), confirm migration/docs/library/<bibkey>.pdf exists. Missing PDF is a hard fail: the project policy (see feedback_no_citation_without_source memory) is that no claim may be cited without a recoverable PDF.

  5. Citation-in-context spot-check. Pick three citations where the paper attributes a specific claim, number, or framing to a cited work. For each:

a. Content accuracy. Grep migration/docs/library/txts/<bibkey>.txt for the most distinctive keyword in the paper's surrounding sentence (e.g. "graduated sanctions", "Seneca cliff"). Open the matching passage in migration/docs/library/<bibkey>.pdf. Confirm the source's actual claim matches what the paper attributes to it — no direction reversal, no overclaim, no understatement. b. Context appropriateness. Confirm the source is being cited at the right place in the argument — i.e., the source's claim actually supports the use the paper is making of it. A citation that is technically a faithful paraphrase but deployed where the source's argument does not apply is a finding (e.g. a study about humans cited to support a claim about LLMs without an explicit bridge). c. Hedging match. If the source itself is hedged ("may", "under certain conditions", "preliminary evidence"), the paper should not present it as an established result. d. Bib-key precision. Grep bdpd_full.bib for the bibkey and confirm year, authors, and venue match the actual PDF — drift between bib entry metadata and PDF metadata is also a finding.

For paper_00 specifically, the load-bearing citations are: Ostrom (graduated sanctions), Bardi (Seneca framework), Hardin (tragedy framing), Fehr-Gächter (altruistic punishment baseline). Spot-check these four first.

  1. Figure reproduce. Pick one figure with a quantitative claim (e.g. the regen × aggressive-count heatmap). Follow the recipe in docs/experiments/reproduce.md to regenerate it from scratch. The reproduce must match the figure in _site/.

  2. Cross-paper consistency. BDPD⁰ is referenced by BDPD¹–³ as the foundational result. Spot-check that BDPD¹'s introduction (paper_01/01-introduction.qmd) characterises BDPD⁰'s findings faithfully. A common drift mode is later papers softening or strengthening earlier claims; this is a finding.

  3. Render check. Run cd migration/docs/publications && bash render_publications.sh --which paper_00 and confirm both HTML and PDF render without unicode breakage ( glyphs in PDF), missing references (?), or LaTeX errors.

  4. Round-2 git-log waypoint. git log d0a26ff..HEAD -- docs/publications/paper_00/ (the round-1 closing commit was 5fa0eaf on 2026-06-11; the round-2 waypoint is d0a26ff on 2026-06-16). The output is a reading-priority hint, not a scope limit — every claim is still re-verified in full.

Round-2 audit dimensions

The five dimensions below augment the core procedure. They are mandatory for round 2. Each has its own Purpose / Procedure / Acceptance block.

Numeric audit (round 2)

Purpose. Re-derive every literal number quoted by the paper from its canonical source, fresh eyes, no trust of the round-1 verdict. This dimension produced the noisiest false positives in round 1 (DS pitch_01: 6/7 invalid findings); the Hints subsection below names every known anchor by path so the trap does not re-trigger.

Procedure.

  1. Numeric ledger. Grep paper_00 qmd for every literal number (regex [0-9]+\.?[0-9]*\s*%? minus stoplist of years, equation labels, figure indices). Build a ledger: claim ↔ paper:line ↔ candidate source.
  2. Source kind: each number is either primary (generated here, anchor = pilot run id + regression baseline + script) or secondary (computed downstream from a paper formula; anchor = the upstream paper-internal value).
  3. Verify primary numbers against data/pilot/<id>/aggregate.json and tests/regression/baseline/. Bit-identity, not "close enough".
  4. Verify recomputable derivations (e.g. "(20.90 − 16.51) / 16.51 = 0.266 ≈ 27%") by hand or python -c. Tag in the ledger.
  5. Bootstrap CIs: any X ± Y or [A, B] must trace to a bootstrap-CI computation in A-B-supplement.qmd (canonical home for paper_00 statistics) or to a script. The bootstrap iteration count and the seed must be documented somewhere — if neither is, flag.

Acceptance.

Check Pass Partial Fail
Ledger coverage every literal traced 1–2 untraced systemic untraced
Primary numbers every primary matches pilot/regression 1 rounding drift magnitude/sign mismatch
Recomputable derivations every derivation reproducible 1 off by stated precision unreproducible
Bootstrap CIs every CI has documented method 1 undocumented systemic

Hints / known anchors (consult Quick reference at bottom of playbook).

Narrative-flow audit (round 2)

Purpose. Round 1 verified that every claim has an evidence chain. Round 2 checks that the claims are delivered as a paper, not as a list of findings: that signposting is consistent, that section-to-section transitions land, and that every claim introduced is also resolved (no orphan setups, no epiphany findings without setup).

Procedure.

  1. Read the paper in one sitting. No backtracking, no jumping to figures. Note every place where you had to re-read a paragraph to keep the thread — those are flow defects.
  2. Claim-thread trace. For each finding in the abstract, locate the section that introduces it, the section that develops it, and the section that resolves it. A finding that appears in Discussion without a Setup section is an orphan; flag it.
  3. Transition audit. Read only the first and last sentence of each section in order. The resulting "skeleton" should still tell the paper's story; if it reads as disjoint, transitions are weak.
  4. Signposting check. Forward references (as we will see in §X) must reach a section that delivers the promised content; backward references (as established in §Y) must reach a section that actually established it, not gestured at it.

Acceptance.

Check Pass Partial Fail
One-sitting readability reads continuous 2–3 re-read spots constant backtracking
Claim-thread completeness every abstract finding traced one orphan/dangler multiple
Skeleton coherence first+last sentences narrate partly nonsense skeleton
Signposting all forward/backward refs land 1–2 broken systemic broken refs

Balance audit (round 2)

Purpose. Round 1 was indifferent to how much space each claim gets. Round 2 checks the proportions: load-bearing findings should occupy the narrative center; secondary findings should not crowd them out; the appendix should not be the home of the paper's most important result.

Procedure.

  1. Section-length census. Count words (or lines) per section. Output a short ranked table.
  2. Claim-importance × length cross-tab. For each abstract finding, compare its prose footprint to its scientific weight. A central finding in 1 paragraph + a secondary finding in 3 pages is an imbalance.
  3. Appendix gravity check. Anything in the appendix that the main text would lose meaning without should be promoted; anything in the main text that is methodological detail without interpretive payload should be demoted. (For paper_00: A-01-platform, A-02-humbaba, A-B-supplement are intended appendices — confirm they hold detail, not result.)
  4. Figure economy. Every figure must earn its space — caption must say what to look at, and the prose nearby must reference it. Figures that exist only because the data exists are candidates for cutting.

Acceptance.

Check Pass Partial Fail
Section length distribution proportional to claim weight 1 outlier multiple
Central finding placement center of paper mostly center buried
Appendix role detail/proof only one finding hides there central result in appendix
Figure economy every figure pulls weight 1 redundant several redundant

Harmonization audit (round 2)

Purpose. The BDPD series has 4 papers + glossary + series_coverage. Round 2 checks that the same construct has the same name across papers, that voice is consistent within paper_00, and that the cross-paper map agrees with the prose.

Procedure.

  1. Glossary check. For every BDPD-specific term in the paper (cliff, collapse, signal-adaptive, rule-bound, mule, pact, ladder, junk arena, meta-agent, world, arena, Forest of Humbaba, …), confirm spelling and meaning match docs/publications/glossary.qmd. Drift is a flag.
  2. Cross-paper naming. Grep the same construct in paper_01, paper_02, paper_03. If paper_00 calls it X and another paper calls it Y, flag.
  3. Voice consistency. A paper should not switch register mid-flow (formal expository → conversational → back). Spot the transitions; flag the largest two.
  4. Cross-paper claim audit. When paper_00 refers forward (paper_01/02/03 builds on this), the forward-paper characterisation in its introduction must match how paper_00 characterises itself. Strengthening or softening a downstream claim is a finding.

Acceptance.

Check Pass Partial Fail
Glossary alignment all terms match 1–2 drift systemic drift
Cross-paper naming one canonical name per construct 1 doublet multiple
Voice consistency uniform 1 switch several switches
Cross-paper claim fidelity downstream-of-self characterisations faithful 1 softening/strengthening multiple

Epistemic-honesty audit (round 2)

Purpose. Catch overclaim, bragging language, unhedged superlatives, mis-attribution of external work. Intellectual honesty is non-negotiable for the series — multiple corrections were already enforced in prior sessions (canonical transparency statement, card-game language softening, "only … we are aware of" hedging). Round 2 sweeps for any remaining or newly-introduced instance.

Procedure.

  1. Superlative scan. Grep paper_00 qmd for \b(first|only|never|every|always|all|none|no one|unique|novel|breakthrough)\b and: keep only when qualified ("to our knowledge", "the dominant tradition", "the corpus we have surveyed", "we are aware of", "we tested", "within the configurations swept").
  2. Card-game language. Confirm zero occurrences of "validates", "verifies", "demonstrates", "proves" applied to the Forest of Humbaba. Replace with "partially reproduces", "cross-checks", "illustrates", "consistency check on a tangible substrate" (see [[bdpd-v2-patterns]] memory).
  3. Transparency statement. Where AI collaboration is mentioned, the canonical form must appear verbatim: "source-grounded by prompt, validated by extensive triaging with different models, integrated under author supervision." No stale variants like "every model output was reviewed", "all decisions verified". (The paper's extended footnote in 00-abstract.qmd reads "…validated by extensive triaging with different models and finally integrated under author supervision"; the "and finally" connective is intentional sequencing prose and is the accepted publication form — not a stale variant.)
  4. Name-drop discipline. Zero non-bibliographic name-drops of any individual as social capital in acknowledgments or cover-letter-style prose — only bibliographic citations are acceptable (see [[no-endorser-mentions]] memory).
  5. Self-cite tone. When paper_00 refers forward to paper_01/02/03, the characterisation must match how the source paper characterises itself — no rhetorical upgrade or downgrade.
  6. External-source hedge match. Every external citation respects the source's own hedging. A meta-analysis with Cohen's d = 0.46 across 50 studies is "found X in a meta-analysis of 50 studies with d = 0.46", not "established that X".

Acceptance.

Check Pass Partial Fail
Superlatives every unhedged removed or qualified 1–2 borderline systemic bragging
Card-game language no validates/verifies/proves 1 slip systemic
Transparency statement canonical wording everywhere 1 stale form systemic stale
Non-biblio name-drops zero 1 (must be removed) multiple
Self-cite tone faithful to source paper framing 1 upgrade/downgrade systemic
Hedge match external citations match source hedge 1 mismatch systemic

Canonical replacements (paste-ready).

  • "BDPD is the only platform" → "the only freely-available platform we are aware of"
  • "every model output was reviewed" → canonical transparency statement (verbatim)
  • "the card game validates the platform" → "the card game partially reproduces … on a tangible substrate"
  • "we are the first to combine X and Y" → "to our knowledge, this is among the first attempts to combine X and Y"
  • "all configurations tested showed Z" → "every configuration we tested showed Z within the swept range"

Acceptance criteria (composite)

Check Pass Partial Fail
Claim ↔ pilot data every claim mapped to a recoverable JSON/log 1–2 unmapped majority unmapped
Claim ↔ code substrate matches paper's description minor parameter drift documented elsewhere code disagrees with paper on a load-bearing constant
Bib audit every cited bibkey has a PDF in library 1–2 missing PDFs (with explanation) ≥3 missing PDFs
Citation accuracy spot-check (3) all three faithful one paraphrase loose any attribution that misrepresents source
Figure reproduce one figure matches paper minor stylistic delta numerical difference
Cross-paper consistency downstream papers cite BDPD⁰ faithfully minor framing shift misrepresentation
Render clean HTML + PDF clean HTML, PDF has stylistic glitch broken render
Numeric audit every literal traced 1–2 untraced systemic
Narrative-flow audit one-sitting readable, no orphans, signposting lands 1 partial check multiple fail
Balance audit sections proportional, central findings at center, figure economy 1 outlier multiple
Harmonization audit glossary aligned, cross-paper naming uniform 1 doublet/drift systemic
Epistemic-honesty audit all six checks pass 1 partial any fail

Common pitfalls

  • Unicode in qmd is forbidden outside math mode (see feedback_no_unicode_in_paper_qmd for the rationale). A ² in body text will render as ? in PDF. Use $^2$ etc.
  • Math in pipe-table cells does not render (feedback_no_math_in_pipe_table_cells). If a reviewer sees a literal $x$ in a PDF table cell, it is a known structural bug class — flag and move on.
  • The bib is shared across all four papers. A reviewer fixing a bib entry for BDPD⁰ should check that the fix does not break BDPD¹–³ rendering.
  • Before the collapse … .epub in library/ is intentional — Bardi popular book, not used for citation, kept for reference.
  • The card-game claims (Tragedy of the Compensator) come from a different experimental layer than the platform claims. The pilot data for these lives under different directories — look for tournament logs, not sweep JSON.
  • Cliff vs collapse — terminology discipline. "Cliff" in the BDPD series is reserved for the Bardi/Seneca asymmetric collapse (slow growth, rapid ruin) — it is BDPD³'s subject, not BDPD⁰'s. paper_00 is about a collapse (any drive of the resource to zero, including symmetric or linear decline). The legitimate uses of "cliff" inside paper_00 are: (a) the localised threshold term "cliff edge" (the value of intensity \(i \approx 0.05\) at which the collapse becomes inevitable — a topological feature of the parameter sweep, not a temporal asymmetry); (b) citations to Bardi/Seneca with the phrase "Seneca cliff". Any other appearance of "cliff" inside paper_00 — describing the finding itself, naming the result informally, or in a section heading — is a terminology drift and a finding. The 2026-06-15/16 sessions corrected several such drifts; round 2 should sweep again to confirm none have crept back. See [[v112-cliff-vs-collapse-audit]] for the canonical distinction.
  • Round-1 false positives — do not re-trigger. DS round-1 audit of pitch_01 generated 6/7 false positives by: (a) searching for cited numbers only in data/pilot/<id>/aggregate.json, ignoring the source paper qmd; (b) conflating cell A/B/C/D semantic identity with legacy pilot dir names (cell_B_on is cell D in the 2×2 factorial, not cell B); (c) demanding standalone computation scripts for effect sizes that are documented in paper qmd with their pooled σ; (d) flagging numbers as "not grounded" without grepping docs/publications/paper_NN/*.qmd first. For every number, consult the Quick reference table below before descending to pilot data.
  • migration/ is the working tree, not the repo root. Path checks fail silently if the reviewer cd's to bdpd/ (parent) and greps there. All playbook paths assume bdpd/migration/ as cwd.

Quick reference — facts you should not re-derive

The following table is the canonical anchor list for paper_00's load-bearing numbers. Every number is locked to its source file:line; reviewers should grep these first before attempting to recompute from pilot data.

Claim Value Canonical anchor
Cliff edge (intensity) \(i \approx 0.05\), 95% bootstrap CI \([0.04, 0.05]\) (10000 iterations) 00-abstract.qmd:14-21, 03-experiments.qmd:106, A-B-supplement.qmd:300
Rescue regen rate \(r \approx 0.95\), bootstrap CI \([0.95, 1.025]\) (fine scan: 0% at \(r=0.925\), 100% at \(r=0.950\)) A-B-supplement.qmd:112-115
Canonical regen default \(r = 0.12\) engine/logistic.js:23 (m.regenRate ?? 0.12)
Slow-regen band ("empirically realistic") \(r \le 0.32\) experiments/definitions/regen_rate_extended.json, heatmap_regen_x_aggr.json
0% gate pass rate (P1–P3 canonical) 0% with 1 aggressive agent 04-discussion.qmd (P1–P3 within canonical regime), tests/regression/baseline/aggressive_fraction.json (gatePassRate=1.0 at n=0, 0.0 at n=1)
Game length collapse 60 → 24 → 10 turns (n=0 → 1 → 6 aggressive) tests/regression/baseline/aggressive_fraction.json:meanTurns
Gini peak \(0.178\) at \(n=1\) aggressive tests/regression/baseline/aggressive_fraction.json:gini.mean
Safe betrayal (CT4) \(+27\%\) = \((20.90 - 16.51) / 16.51 = 0.266\) 00-abstract.qmd:47, 03-experiments.qmd:435-440 (fig-ct4 caption), recomputable from caption
Welfare gain under noise \(63.0 \to 66.4\) (no-noise → 50% noise) 00-abstract.qmd:31-41, 04-discussion.qmd:41-42, tests/regression/baseline/observability_noise.json
Adaptive reduction qualitative; quantified via P8 sweep tests/regression/baseline/adaptive_effectiveness.json, 02-approach.qmd:148-172 (fig-arena-01 caption)
Card-game CT3 zero-effect structural null (CT3 = aggression flag, demo-mode) 03-experiments.qmd:482-487 (fig-ct5b-aggression). --aggression lives only in agents/cards_ai_play.py; demo mode bypasses LLM.
Forest Die threshold 5% 02-approach.qmd:199, agents/cards_ai_play.py

Output

A report at tmp/verification_round2/<reviewer>_paper_00_report.md containing:

  1. Header — reviewer, mode (A/B), commit hash under review, date.
  2. Verified-claim ledger — the list of empirical claims extracted in step 1, each annotated Pass/Partial/Fail with the pilot-data path that backs it (or the Quick-reference row matched).
  3. Bib audit — list of cited bibkeys with PDF present? Y/N and Citation faithful? Y/N/spot-check-only columns.
  4. Numeric ledger — every literal number found in the paper, with anchor (Quick-reference row, pilot data, or recomputation).
  5. Round-2 audit tables — one filled-in table per audit dimension (Numeric / Narrative-flow / Balance / Harmonization / Epistemic-honesty), pass/partial/fail with one-sentence evidence per row.
  6. Findings list — numbered, severity (block / major / minor), location (paper_00/03-experiments.qmd:142 or similar), expected vs observed, suggested fix.
  7. Render notes — any rendering glitch the reviewer encountered.

Reviewer prompt (copy-paste)

You are an independent reviewer asked to verify BDPD⁰ — The foundational paper of the BDPD series, completely and from zero — this is round 2, no round-1 verdict is inherited. The repository is at migration/ on the round-2 working branch (or tag v1.1.2). Follow the playbook at migration/docs/verification/verify_paper_00.md from top to bottom: core Procedure (steps 1–9) plus the five Round-2 audit dimensions (Numeric / Narrative-flow / Balance / Harmonization / Epistemic-honesty). Before starting the Numeric audit, read the Quick reference table at the bottom of the playbook — it lists every load-bearing number with its canonical anchor and is the vaccine against the false-positive pattern that occurred in round 1 (DS pitch_01: six wrong findings from too-narrow grep). The paper's load-bearing claims are listed in the Purpose section. Your job is to confirm that each claim has a recoverable evidence chain (paper text ↔ pilot data ↔ code), that every citation is backed by a PDF in migration/docs/library/, and that the paper passes the five audit dimensions. You should not re-run multi-hour LLM sweeps — the regression baselines under tests/regression/baseline/ already lock the determinism of the substrate. Spend the time on the claim ↔ data trace, the bib audit, and the audit-dimension checks. Produce the report at tmp/verification_round2/<reviewer>_paper_00_report.md (replace <reviewer> with claude, ds, or mimo). Time budget: roughly one and a half working days for the standard round-2 pass.