Skip to content

Verify BDPD² — Governance of nested commons: four vignettes

Purpose

Confirm that BDPD² (paper_02) is internally consistent, empirically backed, and bibliographically clean. The paper's central methodological claim is the governance inversion: across four nested-arena vignettes, the qualitative direction of the governance effect flips sign when rule-bound heuristic agents are replaced with signal-adaptive LLM agents — the same institutional design produces opposite outcomes on the two populations.

Load-bearing items:

  1. The four vignettes — V1 (collapse isolation), V2 (reflexive vs voluntary sanctioner), V3 (world-level treaty enforcer), V4 (junk-arena exclusion).
  2. The inversion pattern — at least one vignette shows a sign flip; the paper claims this is true across every tested vignette (V2/V3/V4, since V1 has no LLM cell by design).
  3. The World layer — treaties, world-level meta-agents, cross-arena linkage. The paper's substrate description must match platform/world.js.

A pass means the seven (vignette, architecture) cells all have recoverable headline statistics, the inversion is documented per vignette, and the substrate description matches code.

Round-2 scope (2026-06-16+). Round 2 is a complete re-verification from zero, not a delta of round 1. Every claim, every citation, every number is re-checked. Round-1 verdicts are inputs only. Five audit dimensions are appended after the core Procedure: Numeric audit, Narrative-flow audit, Balance audit, Harmonization audit, Epistemic-honesty audit. The Quick reference table at the bottom lists every load-bearing number with its canonical anchor. Round-1 reports live at tmp/verification/<reviewer>_paper_02_report.md; round-2 reports land at tmp/verification_round2/<reviewer>_paper_02_report.md (do not overwrite).

Where to look

Paper sources

  • migration/docs/publications/paper_02/index.qmd
  • migration/docs/publications/paper_02/00-abstract.qmd
  • migration/docs/publications/paper_02/01-introduction.qmd
  • migration/docs/publications/paper_02/02-approach.qmd
  • migration/docs/publications/paper_02/03-experiments.qmd (or 03-vignettes.qmd)
  • migration/docs/publications/paper_02/04-discussion.qmd
  • migration/docs/publications/paper_02/_quarto.yml

Compiled output

  • migration/docs/publications/paper_02/_site/index.html and the matching PDF.

Bibliography

  • migration/docs/publications/paper_00/bdpd_full.bib — shared across the four papers.
  • migration/docs/library/<bibkey>.pdf and migration/docs/library/txts/<bibkey>.txt.

Key bibkey families to spot-check:

  • Polycentric governance: ostrom2010polycentric, mcginnis2014social, carlisle2019polycentric, heikkila2018bringing, dorsch2017polycentricclimate, ahlstrom2017governance.
  • Cascade / scale dynamics: mavi2021common, cash2006scale, morrison2019blackbox, volk2012temporal.
  • Hardin / Ostrom framing: hardin1968tragedy, ostrom1990governing.
  • LLM-in-the-loop: park2023generative, gao2023s3, gati2023usingLLMs, mou2026fromindividual, bito2026largelanguagemodelsexhibit, bojic2025politicized.
  • SES robustness / cognitive-science (added 2026-06-11): anderies2004framework, simon1955behavioral, gigerenzer2011heuristic, centola2018experimental.

Pilot data

The paper's actual vignettes (V1–V4) live under the v10_d* series, not the older hardin_3p__* / olson_3p__* directories (those are legacy 3-player pilots not used by paper_02):

  • V1 — World composition (collapse isolation, built-in only): migration/data/pilot/v10_d1/ (per-seed dirs seed_{17,23,29,31,37}/).
  • V2 — Sanctioner ladder (reflexive / voluntary): migration/data/pilot/v10_d3/ (built-in: cell_A_reflex.json, cell_B_voluntary50.json, cell_C_voluntary50_r2.json, comparison.json) and migration/data/pilot/v10_d3_llm_n5/aggregate.json (LLM).
  • V3 — Treaty enforcement: migration/data/pilot/v10_d4/ (built-in) and migration/data/pilot/v10_d4_llm_n5/aggregate.json (LLM); a retained aggregate_n4_archived.json records the historical N=4 aggregate.
  • V4 — Coercive exclusion: migration/data/pilot/v10_d5/ (built-in) and migration/data/pilot/v10_d5_llm_n5/aggregate.json (LLM); same _n4_archived retention pattern.

V1 has no LLM cell by design (paper §3.1 cells-table marks the LLM column as out-of-scope for the substrate-isolation vignette). So the ledger is 7 cells, not 8: 1 (V1 built-in) + 3 × 2 (V2/V3/V4 built-in + LLM).

For V2/V3/V4 the built-in run and the LLM run are the two cells whose comparison produces the inversion claim.

Code — the World substrate

  • platform/world.js — polycentric federation layer and treaty data structures (Treaty class + addTreaty() / removeTreaty() methods are embedded here, not in a separate file; there is no platform/treaties.js).
  • agents/governance.js — world-level meta-agents (WorldSanctionerAgent, TreatyEnforcerAgent, VoluntarySanctionerAgent; PollutionRegulatorAgent is BDPD³ territory and not used in V1–V4).
  • engine/commons.js, engine/metrics.js — shared math primitives, victory metrics.
  • experiments/definitions/*.json and scripts/pilot_v10_d{1,3,4,5}*.mjs — vignette runners that produce the data/pilot/v10_d* aggregates.

Platform documentation (substrate description ↔ code)

  • docs/platform/world.md, docs/platform/worlds.md — the public-facing World API.
  • docs/platform/treaties.md, docs/platform/governance.md — the governance meta-agent contract.

Procedure

  1. Read PDF once, capturing every numerical and directional claim per vignette.
  2. Claim ↔ pilot data trace — for each of the seven (vignette, architecture) cells, confirm the data directory exists and the headline statistic is recoverable.
  3. Inversion check — for each of V2/V3/V4, compute the sign of the effect under built-in agents and under LLM agents. The paper claims a sign flip in all three. A weaker pattern (e.g. flip in 2/3 plus a tie in the third) is a finding.
  4. World ↔ code trace — confirm platform/world.js implements what the paper's substrate description says it implements. Look particularly at cross-arena pollution propagation and meta-agent firing conditions.
  5. Bib audit — restrict to bibkeys cited in paper_02/*.qmd; spot-check the polycentric and cascade literature.
  6. Citation-in-context spot-check. Pick three Ostrom-polycentric citations and three cascade-literature citations. For each:

a. Content accuracy. Grep migration/docs/library/txts/<bibkey>.txt for the most distinctive keyword (e.g. "polycentric", "nested enterprises", "tipping cascade"). Open the matching PDF passage and confirm the source's claim matches what the paper attributes to it. b. Context appropriateness. Polycentric-governance citations describe human institutions; this paper studies LLM agents on a simulated substrate. Confirm each citation is deployed where its empirical substrate is relevant. Citations of human-institution case studies used to support claims about agent behaviour need an explicit bridge. c. Hedging match. Ostrom's design-principle literature is intentionally descriptive (derived from observed long-enduring institutions), not prescriptive. Paper claims should preserve that hedge. d. Bib-key precision. As in verify_paper_00.md step 5.d.

Highest-priority spot-checks: ostrom1990governing, ostrom2010polycentric, carlisle2019polycentric, hardin1968tragedy, olson1965logic, armstrongmckay2022exceeding. 7. Figure reproduce — pick the figure showing the inversion most clearly and regenerate it. 8. Cross-paper consistency — BDPD² is the bridge between BDPD¹ (single arena) and BDPD³ (polycentric Seneca cascade). Confirm BDPD³'s introduction references BDPD²'s polycentric setup faithfully. 9. Render check — bash render_publications.sh --which paper_02. 10. Round-2 git-log waypoint. git log d0a26ff..HEAD -- docs/publications/paper_02/. Reading-priority hint.

Round-2 audit dimensions

Numeric audit (round 2)

Purpose. Re-derive every literal number quoted by paper_02 from its canonical source.

Procedure. Same as paper_00. For V4 the junk-arena purity numbers (1.00 built-in, 0.6 ± 0.3 LLM) are the highest-priority recompute targets.

Acceptance. Same four-row table.

Narrative-flow audit (round 2)

Identical to paper_00. paper_02 specific: the four vignettes must flow as a coherent argument (the inversion across the substrate), not as four disconnected experiments.

Balance audit (round 2)

Identical to paper_00. Confirm each vignette gets proportional treatment; if one vignette dominates 80% of the prose, the inversion claim is structurally lopsided.

Harmonization audit (round 2)

Identical to paper_00. paper_02 specific:

  • The World API names in the paper must match docs/platform/world.md and the code in platform/world.js.
  • Cross-paper: BDPD³'s polycentric setup description must agree with paper_02's substrate description (they share engine/seneca.js and the World layer).

Epistemic-honesty audit (round 2)

Identical to paper_00. paper_02 specific:

  • The inversion claim must be presented as "across the tested vignettes" — not "across every possible governance intervention" (overclaim).
  • The junk-arena LLM purity drop (1.00 → 0.6 ± 0.3) must not be over-narrated as a flaw of LLM agents; the paper's framing is that signal-adaptive agents respond to social signals (conformists follow defectors), which is descriptive, not pejorative.

Acceptance criteria (composite)

Check Pass Partial Fail
7 data cells present all 7 dirs exist with headline JSON 5–6 with explanation for missing ≤4
Inversion in 3/3 vignettes (V2, V3, V4) sign flip in all three flip in 2/3, tie in third < 2/3
World ↔ code substrate description faithful to code minor parameter drift meta-agent firing logic disagrees
Bib audit every cited bibkey present 1–2 missing ≥3 missing
Citation accuracy spot-check (3) all faithful one loose any misrepresentation
Figure reproduce matches paper stylistic delta numerical difference
Render clean HTML + PDF minor glitch broken
Numeric audit every literal traced 1–2 untraced systemic
Narrative-flow audit four vignettes flow as argument one orphan systemic
Balance audit proportional treatment one outlier systemic
Harmonization audit World API ↔ docs ↔ code minor name drift systemic
Epistemic-honesty audit inversion "across tested" hedge present 1 borderline systemic

Common pitfalls

  • Multiple __runN directories per vignette — the paper may aggregate across runs. Confirm the aggregation step matches what the paper reports.
  • The World substrate was new in v1.0 — code paths that don't exist in the v0.x snapshots are not bugs; they are intentional. The paper should be clear about which version it runs on (v1.0+).
  • Pollution propagation between arenas is the cascade mechanism BDPD³ builds on. The propagation coefficient and any decay rate should match between paper_02's substrate description and paper_03's substrate description (they share engine/seneca.js).
  • "Builtin vs LLM" is a population variable, not a code path divergence — the same arena code runs both populations.
  • Cliff vs collapse — terminology discipline. V1 is "collapse isolation", not "cliff isolation" — the BDPD⁰ single-arena finding being replicated is a collapse, not a Seneca cliff. The early pitch_01 deck and some intermediate drafts used "cliff isolation" by mistake (corrected 2026-06-16). Confirm paper_02 uses "collapse isolation" or equivalent collapse-language for V1, and reserves "cliff" / "Seneca cliff" for citations of BDPD³ or Bardi. See [[v112-cliff-vs-collapse-audit]].
  • Round-1 false positives — do not re-trigger. For paper_02 specifically, the inversion claim is qualitative direction, not a numeric delta — do not flag "the numbers don't match" if the direction of the effect is what the paper claims. Confirm direction first, magnitude second.
  • migration/ is the working tree, not the repo root. Path checks fail silently if cwd is bdpd/.

Quick reference — facts you should not re-derive

Claim Value Canonical anchor
V1 — collapse isolation Single-arena collapse replicates across 10 parallel arenas in 1 World; world abstraction does not leak 03-experiments.qmd V1 section, data/pilot/v10_d1/seed_{17,23,29,31,37}/
V2 — visible sanctioner deters Reflexive and voluntary sanctioners both deter LLM defector; delayed sanctioner → collapse 03-experiments.qmd V2 section, data/pilot/v10_d3/{cell_A_reflex,cell_B_voluntary50,cell_C_voluntary50_r2}.json + v10_d3_llm_n5/aggregate.json
V3 — treaty enforcer fires rarely World-level meta-agent fires rarely and that is why the treaty works 03-experiments.qmd V3 section, data/pilot/v10_d4/aggregate.json + v10_d4_llm_n5/aggregate.json
V4 — junk-arena built-in purity \(1.00\) (perfect) 03-experiments.qmd V4 section, data/pilot/v10_d5/aggregate.json
V4 — junk-arena LLM purity \(0.6 \pm 0.3\) — conformists co-breach the pact and follow the defector 03-experiments.qmd V4 section, data/pilot/v10_d5_llm_n5/aggregate.json
Cell count 7 total (V1 has built-in only by design; V2/V3/V4 each have built-in + LLM) 03-experiments.qmd §3.1 cells table
Inversion scope "across the tested vignettes" — V2, V3, V4 (3 inversions, V1 not applicable) 00-abstract.qmd, 04-discussion.qmd
World API canonical names WorldSanctionerAgent, TreatyEnforcerAgent, VoluntarySanctionerAgent (no PollutionRegulatorAgent here — that's BDPD³) agents/governance.js, docs/platform/governance.md
Treaty class location platform/world.js (NOT platform/treaties.js — there is no such file) platform/world.js

Output

A report at tmp/verification_round2/<reviewer>_paper_02_report.md with:

  1. Header (reviewer, mode, commit, date).
  2. 7-cell ledger — one row per (vignette, architecture) cell, with Pass/Partial/Fail and the directory + headline statistic.
  3. Inversion table — 3 rows (V2, V3, V4), with the built-in direction, the LLM direction, and the sign-flip verdict.
  4. Substrate trace — bullets confirming or contradicting each parameter the paper attributes to the World layer.
  5. Bib audit, citation spot-check, render notes.
  6. Round-2 audit tables — one per dimension (Numeric / Narrative-flow / Balance / Harmonization / Epistemic-honesty).
  7. Findings list with severity.

Reviewer prompt (copy-paste)

You are an independent reviewer verifying BDPD² — Governance of nested commons: four vignettes, completely and from zero — this is round 2, no round-1 verdict is inherited. The repository is at migration/ on the round-2 working branch (or tag v1.1.2). Follow the playbook at migration/docs/verification/verify_paper_02.md from top to bottom: core Procedure (steps 1–10) plus the five Round-2 audit dimensions. The paper's central claim is the governance inversion: a sign flip in the governance effect across V2, V3, V4 when LLM agents replace heuristic ones (V1 has no LLM cell by design). Verify the seven (vignette, architecture) cells, the inversion direction per vignette, and that the substrate description matches platform/world.js. Before starting the Numeric audit, read the Quick reference table. Pay particular attention to the cliff-vs-collapse caveat: V1 is "collapse isolation", not "cliff isolation". Produce the report at tmp/verification_round2/<reviewer>_paper_02_report.md. Time budget: roughly one and a half working days.