Skip to content

Experiment Map — Start Here

This page is the entry point for anyone new to the BDPD experiments. It answers three questions in order:

  1. What categories of experiment exist? — from a one-line deterministic parameter sweep to a full LLM arena with cheap-talk, pacts and sanctions.
  2. How do I run one, and what does it cost?
  3. How do I turn one into my own variant?

If you only want to watch an experiment rather than run it, jump to Watch instead of run.

For the metrics and statistical protocol behind the numbers see Protocol & Metrics; for copy-paste reproduction commands see Reproducing Results.

The four tiers, simplest to most complete

Tier Category Instrument LLM? Cost What it exercises
A Builtin parameter sweeps (P1–P11) Platform No Free, seconds–minutes Heuristic agents only: phase boundaries, regen, pool size, perturbations
B Supplement sweeps (B1–B5) Platform No Free Cliff-edge intensity, extended regen rescue, cross-sweep inference
C Card tournaments (CT1–CT6) Card game CT6 only CT6 costs tokens Discrete, stochastic archetype matchups
D LLM governance pilots (C/D/S series) Platform + real LLM Yes DeepSeek tokens (~$0.15–0.45/pilot) Cheap-talk, pacts, graduated sanctions, governance meta-agents, Seneca cascade — the advanced mechanisms, under real model cognition

Tiers A–C are the bulk of the published statistics (deterministic and cheap to reproduce). Tier D is where the advanced social mechanisms live — and is the part newcomers most often miss, because the pilots are standalone scripts in scripts/, not JSON sweep definitions.

At a glance: mechanism x experiment x paper x replay

One row per advanced mechanism, linking the code that runs it, the paper it appears in, the recorded game that shows it, and the concept doc that explains it. Papers use the BDPD⁰–³ naming and are pinned to a release tag (see Reproducing Results → Pin a version).

Mechanism Run it (script) Paper (tag) Watch it (replay) Read it (concept)
Cheap-talk (announce intended harvest) pilot_d1_n5.mjs, pilot_d1_cell_c.mjs BDPD¹ (v0.9) llm_cheap_talk Cheap Talk
Sanctions (perturbation on violation) pilot_c1.mjs BDPD¹ (v0.9) llm_graduated Governance, Metrics
Graduated sanctions (penalty ladder) pilot_d2_ladder_sweep.mjs BDPD¹ (v0.9) llm_graduated Governance
Pacts (binding agreements) pilot_v11_s1_seneca_standalone.mjs, smoke_pacts.mjs BDPD¹ (v0.9) llm_graduated Pacts
Governance meta-agents (treaty enforcer → BDPD²; pollution regulator → BDPD³) pilot_v10_d4_treaty_enforcer.mjs, pilot_v11_s3_pollution_governance.mjs BDPD² (v1.0), BDPD³ (v1.1) s3_cap_leading, s3_ungoverned Governance, Treaties
Seneca pollution cascade pilot_v11_s2_polycentric.mjs BDPD³ (v1.1) s2_cascade Perturbations, Engine System
All mechanisms together (showcase) — (recorded run) — llm_cascade Replay Player

The "Watch it" column names a curated fixture served by the Replay Player — no API key or compute needed.

Tier A & B — builtin sweeps (no LLM, free)

Deterministic given a seed; no API key needed. Defined as JSON in experiments/definitions/ and run through one CLI:

node experiments/experiment.js list                     # all sweep definitions
node experiments/experiment.js run aggressive_fraction   # P1
node experiments/experiment.js run perturbation_regen_shock  # P10

Tier C — card tournaments

Discrete game, higher stochastic variance (200 games/cell). CT1–CT5 use heuristic decks; CT6 is the qualitative LLM case study.

Gentlest on-ramp. To interpret a BDPD run against a familiar reference point, see the worked replications of classic commons literature in the repo under examples/scenarios/ (Hardin 1968, Olson 1965, Janssen & Ostrom 2006, Dayton-Johnson 2002) — each with its own README and reproducibility notes. Index: examples/README.md.

Tier D — LLM governance pilots (the advanced mechanisms)

These run a real LLM (DeepSeek-flash by default) inside the platform arena, so the agents actually decide to talk, promise, comply with a cap, or defect. Each pilot is a self-contained script in scripts/ with a header documenting its question and exact run command; it needs an API key and costs tokens (each script estimates its own cost and skips cleanly if no key is set). The matrix above maps every mechanism to its script, paper and replay fixture; full commands, costs and run times are in Reproducing Results, organised by release (v0.9 governance pilots, v1.0 polycentric pilots, v1.1 Seneca experiments).

Smoke vs pilot. The smoke_* scripts in scripts/ (smoke_governance_arena.mjs, smoke_pacts.mjs, smoke_c1.mjs, smoke_b1c2.py, ...) are fast, free, no-LLM functional checks of the same mechanisms — run these first to see the plumbing work before spending tokens on a pilot_* run.

Make your own variant

There are three levels of variation, from cheapest to deepest.

1. Vary a builtin sweep (Tier A/B)

Copy a definition, edit the swept values, run it. No code changes.

cp experiments/definitions/aggressive_fraction.json \
   experiments/definitions/my_variant.json
# edit the sweep axis / agent mix in my_variant.json
node experiments/experiment.js run my_variant

The schema (axes, agent mix, seeds, runs) is documented in Definition Format.

2. Vary an LLM pilot (Tier D)

Each pilot_*.mjs exposes its knobs as consts at the top of the file — edit them and re-run. For example scripts/pilot_d1_n5.mjs:

const SEEDS     = [17, 23, 29, 31, 37];   // add/remove seeds (N)
const MAX_TURNS = 30;                       // game length
const N_AGENTS  = 6;                        // pool size

The behaviour of each LLM player is controlled by flags on the agent bridge agents/bdpd_agent.py (the pilots spawn it as a subprocess):

Flag Effect
--archetype {aggressive,conservative,adaptive,random} Persona nudge prepended to the system prompt
--no-talk Disables the cheap-talk channel (tools + announced field) — the D1 Cell C manipulation
--no-memory Bare-number responses, no scratchpad
--noisy-observations Commons stock reported with noise
--temperature, --max-tokens, --max-history, --model-name, --model-url Standard inference / model selection

See Agent Types → LLM Agents and the Agent SDK for the bridge contract.

3. Vary the prompts themselves

The prompt text is not in these docs by design — the code is the source of truth. To change what an agent is told, edit:

  • agents/bdpd_agent.py — BASE_GAME_RULES, ARCHETYPE_NUDGES, JSON_RESPONSE_INSTRUCTION / ..._NO_TALK, assembled by get_system_prompt(); per-turn user text in obs_to_text().
  • agents/bdpd_tools.py — make_action_tools() (the broadcast, send_private, announce_intended_harvest, propose_pact, accept_pact, pledge tool descriptions the model reads).
  • agents/cards_ai_play.py — build_system_prompt() for the card game.

The docs paraphrase these; see Prompt Design → Source of truth.

Watch instead of run

You do not need an API key — or any compute — to see the advanced experiments. A curated, version-controlled set of recorded games ships in the repo and is served by the Replay Player:

node main.js     # then open http://localhost:3000/replay/replay.html

The catalogue includes the advanced mechanisms end-to-end:

Fixture Shows
llm_cascade LLM governance + cascade — every mechanism, real cognition (drill into arenas A/B/C)
llm_graduated Cheap-talk + pacts + graduated sanctions in one arena
llm_cheap_talk Players announcing harvests over the cheap-talk channel
s3_cap_leading / s3_ungoverned Governed vs ungoverned pollution cascade
s2_cascade The "no visible culprit" pollution signal

These are generated from runtime data by scripts/curate_fixtures.mjs into web/replay/fixtures/. Re-run that script to refresh the showcase after generating new traces.