Reproducing Results¶

All BDPD experiments are deterministic given a seed. This page provides the exact commands to reproduce every published result.

Prerequisites¶

# Complete setup
./scripts/setup.sh
source venv_py_bdpd/bin/activate

# Verify environment
./scripts/check_env.sh

Pin a version¶

Numerical results in the published papers are pinned to specific tags:

Paper	Tag	Notes
BDPD⁰ — platform sweeps P1–P11, card tournaments CT1–CT5 (+ CT6 qualitative)	`v0.4.1` (experiments) / `v0.5.1` (code extended)	Logistic engine canonical baseline
BDPD¹ — C1, D1, D2, D3 governance pilots	`v0.9`	Cheap-talk, sanctions, LLM agents
BDPD² — V1–V4 polycentric vignettes	`v1.0`	World API, links, treaties
BDPD³ — S0–S3 Seneca	`v1.1`	Seneca engine + pollution links

To reproduce a paper's numbers, check out the matching tag before running the commands below:

git fetch --tags
git checkout v1.1            # or v1.0 / v0.9 / v0.5.1 / v0.4.1

The main branch tracks ongoing work and may diverge from published results — including from genuine bug fixes that legitimately change numbers (e.g. the v1.1 ConservativeAgent hidden-stock fix re-pivots the P9 "hidden" point of observability_noise).

Reproducing Platform Sweeps¶

Via the Experiment Framework¶

# P1  — Aggressive fraction sweep
node experiments/experiment.js run aggressive_fraction

# P2  — Regen rate sweep
node experiments/experiment.js run regen_rate_sweep

# P3  — Pool size scaling
node experiments/experiment.js run pool_size_scaling

# P4  — Pool size fine
node experiments/experiment.js run pool_size_fine

# P5  — 2D parametric map
node experiments/experiment.js run heatmap_regen_x_aggr

# P6  — Asymmetry sweep
node experiments/experiment.js run asymmetry_sweep

# P7  — Scheduler comparison
node experiments/experiment.js run scheduler_comparison

# P8  — Adaptive effectiveness
node experiments/experiment.js run adaptive_effectiveness

# P9  — Observability noise
node experiments/experiment.js run observability_noise

# P10 — Regen shock
node experiments/experiment.js run perturbation_regen_shock

# P11 — Mule strategy override
node experiments/experiment.js run mule_strategy_override

Supplement sweeps (B-series)¶

# B1  — Aggressor intensity (coarse)
node experiments/experiment.js run aggressive_intensity_sweep

# B1f — Aggressor intensity (fine)
node experiments/experiment.js run aggressive_intensity_sweep_fine

# B2  — Adaptive reduction sweep
node experiments/experiment.js run adaptive_reduction_sweep

# B3  — Regen rate extended (coarse)
node experiments/experiment.js run regen_rate_extended

# B3f — Regen rate extended (fine)
node experiments/experiment.js run regen_rate_extended_fine

# B4  — Pure Seneca shock
node experiments/experiment.js run pure_seneca_shock

Via Python Sweep Scripts¶

# Plot canonical sweeps (reads result JSONs, not definitions)
python3 experiments/sweep.py --latest

# Plot with specific theme
python3 experiments/sweep.py --latest --theme light_paper

# Plot supplement sweeps
python3 experiments/sweep_supplement.py --latest

Reproducing Card Tournaments¶

# CT1 — Collapse-rate matrix (200 games per matchup)
python3 experiments/sweep_card_tournaments.py \
  --cards cards/cards_v03.json \
  --sweep ct1

# CT2 — Game-length sweep
python3 experiments/sweep_card_tournaments.py \
  --cards cards/cards_v03.json \
  --sweep ct2

# CT3 — Observability noise
python3 experiments/sweep_card_tournaments.py \
  --cards cards/cards_v03.json \
  --sweep ct3

# CT4 — Defection timing
python3 experiments/sweep_card_tournaments.py \
  --cards cards/cards_v03.json \
  --sweep ct4

# CT5 — Stranger-King deep dive
python3 experiments/sweep_card_tournaments.py \
  --cards cards/cards_v03.json \
  --sweep ct5

CT6 ("LLM awareness-without-restraint") is a qualitative finding, not a parametric sweep: it is observed in LLM-mode game transcripts produced by agents/cards_ai_play.py (the LLM keeps harvesting while reporting that the forest is collapsing). It has no --sweep ct6 recipe and no aggregate data file; reproduce it by reading the LLM play logs, not via sweep_card_tournaments.py.

Quick Demo (Reduced Runs)¶

# CT1 with 20 games (for rapid iteration)
python3 experiments/sweep_card_tournaments.py \
  --cards cards/cards_v03.json \
  --sweep ct1 --demo

Reproducing v0.8 LLM Pilot¶

The original 6-cell LLM pilot (3 mixes × 2 scenarios, DeepSeek v4-flash agents via the Python middle-man SDK).

# Full 6-cell run (~$2–3 DeepSeek)
DEEPSEEK_API_KEY=sk-... python3 examples/pilot/run_pilot.py

# Dry-run: 1 cell, maxTurns=3 (verify wiring, minimal cost)
DEEPSEEK_API_KEY=sk-... python3 examples/pilot/run_pilot.py --dry-run

Output: data/pilot/<scenario>__<mix>/ per cell, data/pilot/pilot_summary.json aggregate.

Reproducing v0.9 Governance Pilots (BDPD¹ paper)¶

These run as standalone Node scripts against a live platform instance. Start the platform first (node main.js &).

The four mini-pilots in BDPD¹ (C1, D1, D2, D3) all run at \(N = 5\) seeds with DeepSeek-flash LLM agents. C1 defaults to the legacy set [17, 42, 123, 456, 789] (overridable with PILOT_SEEDS=17,42 for a faster smoke run); D1/D2/D3 use the project anchor set [17, 23, 29, 31, 37] (5 primes including the BDPD⁰ anchor seed 17).

Builtin smoke (no LLM cost, mechanism only)¶

# B3 — Hardin 3-player announce pilot (honour rate, silent defection)
node scripts/smoke_b3.mjs

LLM mini-pilots (require `DEEPSEEK_API_KEY`)¶

# B1d — Cheap-talk pilot (3 players, 7 turns, announce_intended_harvest)
python3 scripts/smoke_b1d_pilot.py

# C1 — Sanction mechanism, 3-player N=5 (~$0.25 DeepSeek, ~225s)
DEEPSEEK_API_KEY=sk-... node scripts/pilot_c1.mjs

# D1 — Cheap-talk cliff, 2×2 factorial × N=5 seeds (~$0.30 DeepSeek)
DEEPSEEK_API_KEY=sk-... node scripts/pilot_d1_n5.mjs

# D1 Cell C — LLM no-talk control (4/5 preservation without talk)
DEEPSEEK_API_KEY=sk-... node scripts/pilot_d1_cell_c.mjs

# D2 — Mule graduated vs constant, single-arena 5-agent × N=5 (~$0.15)
DEEPSEEK_API_KEY=sk-... node scripts/pilot_d2_llm_n5.mjs

# D3 — Ladder shape sweep (soft / hard / flat-5) × N=5 (~$0.45)
DEEPSEEK_API_KEY=sk-... node scripts/pilot_d2_ladder_sweep.mjs

Output lands in data/pilot/{c1,d1_n5,d1_cell_c,d2_llm_n5,d2_ladder_sweep}/ with per-seed subdirectories seed_<S>/ and a canonical aggregate.json (see aggregate schema below).

Reproducing v1.0 Polycentric Pilots¶

Builtin-only¶

# D1 — Cliff isolation across 10 arenas in one World (no links/treaties)
node scripts/pilot_v10_d1_cliff_polycentric.mjs

# D2 — Mule with graduated sanctions in polycentric setup
node scripts/pilot_v10_d2_mule_polycentric.mjs

# D3 — Yoon voluntary sanctioner (discretion vs reflexive)
node scripts/pilot_v10_d3_yoon_voluntary.mjs

# D4 — Cross-arena treaty enforcer
node scripts/pilot_v10_d4_treaty_enforcer.mjs

# D5 — Multi-source exclusion sweep
node scripts/pilot_v10_d5_exclusion_sweep.mjs

LLM pilots — paper BDPD² vignettes V2/V3/V4 (require `DEEPSEEK_API_KEY`)¶

# V2 — D3 voluntary sanctioner ladder under LLM agents, N=5 (~$0.30)
DEEPSEEK_API_KEY=sk-... node scripts/pilot_v10_d3_llm_n5.mjs

# V3 — D4 treaty enforcer under LLM agents, N=5 (~$0.40)
DEEPSEEK_API_KEY=sk-... node scripts/pilot_v10_d4_llm_n5.mjs

# V4 — D5 exclude perturbation under LLM agents, N=5 (~$0.45)
DEEPSEEK_API_KEY=sk-... node scripts/pilot_v10_d5_llm_n5.mjs

Output: data/pilot/v10_d{3,4,5}_llm_n5/seed_<S>/ + aggregate.json. Costs are realised DeepSeek-flash prices from paper_02 A1 §LLM cost calibration (tbl-cost); the list-price upper bound is ~6× higher.

Note. Single-arena mule LLM pilots (D2 / D3 ladder sweep) belong to the BDPD¹ paper, not BDPD² — they are listed in the v0.9 section above. The polycentric counterparts (mule / treaty / exclude across a world of arenas) are the pilot_v10_* scripts in this section.

Reproducing v1.1 Seneca Experiments¶

S0 — Agent calibration¶

# Sweep baseFraction for RCP agents on Seneca (find ≈0.035)
node scripts/calibration_v11_s0_seneca.mjs

Output: data/calibration/v11_s0_seneca/aggregate.json.

S1 — Seneca standalone (mule, pact, no governance)¶

node scripts/pilot_v11_s1_seneca_standalone.mjs

S2 — Polycentric cascade (pollution links)¶

node scripts/pilot_v11_s2_polycentric.mjs

S3 — Pollution governance (leading vs lagging trigger)¶

# Main pilot
node scripts/pilot_v11_s3_pollution_governance.mjs

# OFAT structural robustness (k1, baseFraction, linkRate, dt × 5 levels)
node scripts/robust_v11_s3_ofat.mjs

# Plot S3 figures (light_paper theme for publication)
python3 scripts/plot_v11_s3.py

Seneca ODE validation (standalone, no platform)¶

python3 experiments/seneca_validate.py --all

Reproducing Publication Figures¶

Unified runner¶

run_all.sh runs experiments and generates P-sweep / B-sweep / Seneca ODE figures:

# Full pipeline: experiments + figures
experiments/run_all.sh

# Figures only (from existing result JSONs)
experiments/run_all.sh --plots-only

# Selective
experiments/run_all.sh --group seneca    # S1 ODE figures only
experiments/run_all.sh --only P1 P2 B3   # specific sweeps

Per-paper plot scripts¶

These read from data/pilot/ and write PNG figures. Run them after the corresponding pilot has completed.

# Paper 01 — signal-adaptive figure
python3 scripts/plot_signal_adaptive.py

# Paper 01 — V2 inversion (builtin vs LLM across sanctioner policies)
python3 scripts/plot_v2_inversion.py

# Paper 02 — D1 cheap-talk (requires D1 N=5 data)
python3 scripts/plot_d1_n5.py

# Paper 02 — D2 mule ladder (requires D2 ladder sweep data)
python3 scripts/plot_d2_ladder_sweep.py

# Paper 02 — D2 LLM graduated vs constant (requires D2-LLM N=5 data)
python3 scripts/plot_d2_llm_n5.py

# Paper 03 — S3 pollution governance
python3 scripts/plot_v11_s3.py

Single-seed plot variants (plot_d1.py, plot_d2.py, plot_d2_llm.py) exist for development; the _n5 versions above are the publication-ready counterparts.

Smoke tests

Scripts under scripts/smoke_*.mjs and scripts/smoke_*.py are development validation tests (wiring, protocol, cache behaviour), not experiment reproductions. They are not listed here.

Seed Configuration¶

Seeds are embedded in each experiment definition JSON under experiments/definitions/. The card tournament runner accepts a --seed flag:

python3 experiments/sweep_card_tournaments.py \
  --cards cards/cards_v03.json \
  --sweep ct1 --seed 42

For reproducible LLM experiments, use temperature=0.0 and a fixed seed:

python agents/cards_ai_play.py \
  --deck1 stranger --deck2 warrior \
  --cards cards/cards_v03.json \
  --games 20 --seed 42

Output Locations¶

Output	Path
Platform sweep results	`experiments/results/<definition>.json`
Platform sweep figures	`experiments/figures/platform/`, `experiments/figures/cards/`
Seneca ODE figures	`experiments/figures/seneca/`
Seneca validation figures	`experiments/fig_*__<theme>.png`
Card tournament results	`results/<timestamp>_<sweep>/`
v0.8 LLM pilot	`data/pilot/<scenario>__<mix>/`
v0.9 governance pilots	`data/pilot/{b3,c1,d1,d1_n5,…}/`
v1.0 polycentric pilots	`data/pilot/{v10_d1,v10_d2,v10_d3,…}/`
v1.1 Seneca pilots	`data/pilot/{v11_s1,v11_s2,v11_s3}/`
v1.1 calibration	`data/calibration/v11_s0_seneca/`
v1.1 robustness	`data/robust/v11_s3/`
Per-paper figures	`scripts/` working dir (PNG)
Statistical report	`experiments/results/B5_statistical_report.md`

Aggregate schema convention¶

Multi-seed pilots write per-seed runs under data/pilot/<pilot>/seed_<S>/ and a top-level data/pilot/<pilot>/aggregate.json. The aggregate is the canonical input for plot scripts and paper tables — single-seed JSONs are kept for debugging and replay only.

Aggregate shape (illustrative, fields vary slightly per pilot):

{
  "pilot":   "d2_llm_n5",
  "seeds":   [17, 42, 123, 456, 789],
  "perSeed": [ { "seed": 17, /* cell metrics */ }, … ],
  "summary": {
    "<metric>": { "mean": ..., "sd": ..., "min": ..., "max": ... }
  }
}

If both aggregate.json and aggregate_N5.json exist locally, the canonical file is aggregate.json. Earlier drafts of v1.0 / v0.9 left both around when a seed was retroactively dropped or added; the audit-47 batch (commit 5503252) consolidates to a single canonical file per pilot. Re-running the pilot regenerates the canonical file in place; data/pilot/ is gitignored so artefacts are reproducible but not tracked.

Historical _n4_archived retention (V3 / V4 LLM pilots). Two specific directories — data/pilot/v10_d4_llm_n5/ and data/pilot/v10_d5_llm_n5/ — additionally retain an aggregate_n4_archived.json alongside the canonical N=5 aggregate.json. This is the historical N=4 aggregate produced before the N=5 cell was added during the v1.0 paper_02 rewrite; it predates the audit-47 consolidation and is not consulted by any current script. It is safe to ignore unless you are tracing the v0.9 → v1.0 lineage. If you regenerate aggregate.json from scratch, leave the _n4_archived file untouched.

v11_s3 robustness output (data/robust/v11_s3/ofat.json) follows the same one-file-per-pilot convention; it has no per-seed subdirectories because Seneca is deterministic (the OFAT script perturbs structural parameters one at a time, not seeds — see S3).

Visualisation¶

# Generate figures with dark theme
python3 experiments/sweep.py --latest

# Generate figures with light paper theme (for publications)
python3 experiments/sweep.py --latest --theme light_paper

Available themes via plot_theme.py: - dark (default) — dark background, coloured accents - light_paper — white background, high-contrast for publication - solarized — muted palette

Expected Output¶

For P1 (aggressive fraction sweep), expect:

aggressive_count  gate_pass  game_length   gini   welfare
0                 1.00       60.0          0.00    86.41
1                 0.00       24.0          0.18    55.37
2                 0.00       17.0          0.16    53.10
3                 0.00       14.0          0.14    53.71
4                 0.00       12.0          0.10    54.94
5                 0.00       11.0          0.05    57.40
6                 0.00       10.0          0.00    59.94

The discontinuous threshold between 0 and 1 aggressive agents should be immediately visible: gate pass drops from 100% to 0%, and game length contracts monotonically as more aggressors accelerate collapse (60 → 24 → … → 10 turns).

For CT1 (collapse-rate matrix), the Warrior-King row should show the highest collapse rates (54.5%–81.5%), and the Temple-Keeper column the lowest (0%–54.5%).

Statistical Verification¶

To replicate the cross-sweep statistical inference (B5):

python3 experiments/statistical_inference.py \
  --results-dir experiments/results_paper/platform \
  --report experiments/results/B5_statistical_report.md \
  --bootstrap-n 10000 \
  --permutation-n 10000 \
  --seed 42

This produces the logistic regression table, bootstrap CIs for threshold locations (B1, B3 use fine sweeps when available), and permutation test results for P7. The --seed flag controls the random seed for all bootstrap and permutation tests (default: 42).

Reproducing Results¶

Prerequisites¶

Pin a version¶

Reproducing Platform Sweeps¶

Via the Experiment Framework¶

Supplement sweeps (B-series)¶

Via Python Sweep Scripts¶

Reproducing Card Tournaments¶

Quick Demo (Reduced Runs)¶

Reproducing v0.8 LLM Pilot¶

Reproducing v0.9 Governance Pilots (BDPD¹ paper)¶

Builtin smoke (no LLM cost, mechanism only)¶

LLM mini-pilots (require DEEPSEEK_API_KEY)¶

Reproducing v1.0 Polycentric Pilots¶

Builtin-only¶

LLM pilots — paper BDPD² vignettes V2/V3/V4 (require DEEPSEEK_API_KEY)¶

Reproducing v1.1 Seneca Experiments¶

S0 — Agent calibration¶

S1 — Seneca standalone (mule, pact, no governance)¶

S2 — Polycentric cascade (pollution links)¶

S3 — Pollution governance (leading vs lagging trigger)¶

Seneca ODE validation (standalone, no platform)¶

Reproducing Publication Figures¶

Unified runner¶

Per-paper plot scripts¶

Seed Configuration¶

Output Locations¶

Aggregate schema convention¶

Visualisation¶

Expected Output¶

Statistical Verification¶

LLM mini-pilots (require `DEEPSEEK_API_KEY`)¶

LLM pilots — paper BDPD² vignettes V2/V3/V4 (require `DEEPSEEK_API_KEY`)¶