Quickstart — Design an experiment

You have a question about commons dynamics — does X collapse the system?, how does Y interact with Z? — and you want a clean answer with error bars, not a single anecdotal run. This guide walks the path from question to publishable sweep, in three layers of depth.

The three artifacts of a BDPD experiment:

• An arena spec — the world your agents inhabit (engine, commons parameters, observability, scheduler).
• A sweep axis — the one thing you vary, with the rest held fixed.
• Metrics — what you read off at the end of each run.

Optional fourth artifact: a perturbation — a structured mid-game shock (regen change, wealth shock, strategy override) scheduled at a trigger.

Prerequisites

Complete the installation first. Activate the venv: source venv_py_bdpd/bin/activate.

---

Layer 1 — The smallest experiment (one arena via curl)

Before designing a sweep, get one arena to run end-to-end. This is the eyeball-check layer: useful for sanity, not for inference.

Start the platform:

node main.js

Create an arena, capture its keys in shell variables:

RESPONSE=$(curl -s -X POST http://localhost:3000/api/v1/arenas \
  -H "Content-Type: application/json" -d '{
    "model": {
      "engineMode":      "logistic",
      "commonsInitial":  150,
      "commonsCapacity": 150,
      "regenRate":       0.12,
      "maxTurns":        60,
      "threshold":       10,
      "scheduler":       "simultaneous",
      "tickInterval":    null,
      "minPlayers":      2,
      "maxPlayers":      4
    },
    "observability": {
      "commonsStock":  { "visible": true, "noise": 0.05 },
      "regenRate":     { "visible": false },
      "othersWealth":  { "visible": true, "noise": 0.10 },
      "othersHarvest": { "visible": true, "noise": 0    }
    }
  }')
ARENA_ID=$(echo "$RESPONSE" | jq -r .arenaId)
OWNER_KEY=$(echo "$RESPONSE" | jq -r .ownerKey)
ARENA_KEY=$(echo "$RESPONSE" | jq -r .arenaKey)

Join two agents — built-in strategies are aggressive, conservative, adaptive, rcp (Seneca mode), random:

for s in aggressive conservative; do
  curl -s -X POST "http://localhost:3000/api/v1/arenas/${ARENA_ID}/join" \
    -H "Content-Type: application/json" -H "X-Arena-Key: ${ARENA_KEY}" \
    -d "{\"agentType\":\"builtin\",\"strategy\":\"${s}\",\"privateResource\":30}"
done

Start and tick to completion (with tickInterval: null, you advance turns manually):

curl -s -X POST "http://localhost:3000/api/v1/arenas/${ARENA_ID}/start" \
  -H "X-Owner-Key: ${OWNER_KEY}"
while true; do
  RESULT=$(curl -s -X POST "http://localhost:3000/api/v1/arenas/${ARENA_ID}/tick" \
    -H "X-Owner-Key: ${OWNER_KEY}")
  echo "$RESULT" | jq -r '.state // "done"' | grep -q done && break
done

curl -s "http://localhost:3000/api/v1/arenas/${ARENA_ID}/summary" \
  -H "X-Owner-Key: ${OWNER_KEY}" | jq '.summary | {turns, victory, metrics}'

Useful fields under summary:

Field	Meaning
victory.gatePassed	Commons survived above threshold
turns	Turns played before stop condition
metrics.welfareScore	Sen-type welfare (mean × (1 − Gini))
metrics.gini	Final wealth inequality
commons.stock	Remaining commons stock

This run is one sample. The next layer turns it into a question with statistics.

---

Layer 2 — From one arena to a sweep

A sweep is the same arena spec replayed across one varying axis, many times per cell. BDPD's sweep runner reads a definition JSON and produces a results JSON suitable for plotting and bootstrap CIs.

Definitions live in experiments/definitions/. Start by copying the closest match — for an "agent composition" question, that's aggressive_fraction.json:

cp experiments/definitions/aggressive_fraction.json \
   experiments/definitions/my_first_sweep.json

The four blocks of any definition:

{
  "name": "my_first_sweep",
  "description": "Short prose so future-you remembers the question.",

  "baseConfig":   { /* arena model — same fields as the curl spec */ },
  "observability":{ /* same shape as the curl spec */ },
  "baseAgents":   [ /* fixed roster, with stable IDs a0, a1, ... */ ],

  "sweep": {
    "axis":   "what is varying (free-form label for plots)",
    "points": [
      { "label": "...", "axisValue": <number>,
        "agentOverrides":   [ /* swap a strategy on a specific id */ ],
        "configOverrides":  { /* override baseConfig fields */ },
        "_perturbations":   [ /* see Layer 3 */ ] }
    ]
  },

  "runsPerPoint": 20,
  "metrics": ["turns", "metrics.gini", "metrics.welfareScore", "metrics.commonsRatio"]
}

Three rules that catch most mistakes:

1. Stable IDs, a0..aN. agentOverrides references them by id — reorder the array and your sweep silently changes meaning.
2. runsPerPoint matters only for stochastic dimensions (observability noise, scheduler "wealth_weighted", LLM agents). For deterministic sweeps it can be 1; inference is at the between-cell level via bootstrap (see docs/experiments/index.md for the statistical procedure).
3. One axis per definition. A 2D map (e.g. P5) is structured as a single sweep whose points are (r × aggressive_count) pairs, not nested sweeps. The plotter understands axisValue as a numeric key; for 2D, encode the second dimension in label and slice in Python.

List and inspect what's available:

node experiments/experiment.js list
node experiments/experiment.js show aggressive_fraction

Run your sweep:

node experiments/experiment.js run my_first_sweep --out experiments/results

The runner instantiates Arena objects in-process — no HTTP server required, no curl loop, no race conditions. Output is one timestamped JSON in --out, plus a sibling .manifest.json with content-addressed hashes for provenance (node tools/manifest.mjs verify <manifest>).

---

Layer 3 — Add a perturbation

A perturbation is a scheduled change to arena state mid-run. Twelve built-in types in v1.1 (registry in platform/perturbation.js):

Type	Payload	What it does
regen_shock	{ factor }	Multiply regen rate (e.g. 0.25 = severe decline)
regen_set	{ value }	Set regen rate absolute
commons_shock	{ delta }	Add/remove stock from commons
capital_shock	{ factor?, delta? }	Seneca-only: shock capital stock \(C\)
capacity_shock	{ factor }	Multiply carrying capacity
wealth_shock	{ factor?, delta?, playerId? }	Per-player or global wealth change
strategy_override	{ playerId, strategy }	Swap a builtin's strategy mid-game
observability_flip	{ variable, visible }	Toggle observability of a variable
threshold_shift	{ newThreshold }	Change collapse threshold
sanction	{ playerId, amount }	Flat fine on a violator
sanction_graduated	{ playerId, ladder }	Ostrom-style escalating ladder
exclude	{ playerId, duration? }	Remove access to the commons for N turns

A trigger fires the perturbation: { kind: "turn", turn: 20 } is the common case; { kind: "threshold", variable: "commonsRatio", below: 0.3 } fires endogenously.

To sweep the strength of a regen shock at turn 20 — the design behind P10 in the paper — copy perturbation_regen_shock.json and edit the points. The shape per point:

{ "label":     "shock ×0.25",
  "axisValue": 0.25,
  "_perturbations": [
    { "id":      "s1",
      "type":    "regen_shock",
      "trigger": { "kind": "turn", "turn": 20 },
      "payload": { "factor": 0.25 },
      "description": "Heavy regen decline at turn 20" }
  ] }

Multiple perturbations in one run are allowed — they fire independently. A "no perturbation" point is "_perturbations": [].

Custom perturbation types are plugins: see Extend the engine and the starter at examples/plugins/perturbations/.

---

Layer 4 — Plot and read the results

python experiments/sweep.py --latest --out experiments/figures --theme all

--latest picks up the most recent result JSON in experiments/results/. --theme all produces both the dark variant (for the docs site) and _light_paper (for publication).

For statistical inference beyond plotting — bootstrap CIs on phase boundaries, permutation tests for scheduler effects — see docs/experiments/index.md for the methodology and reproduce.md for the exact commands used in the paper.

---

Common patterns

Composition sweep. Vary the strategy mix at fixed pool size. Template: aggressive_fraction.json. Use agentOverrides to swap identities; keep baseAgents as the most cooperative baseline so deltas are interpretable.

Parameter sweep. Vary one number in baseConfig (regen rate, pool size, threshold). Template: regen_rate_sweep.json. Use configOverrides per point.

Shock sweep. Hold the world fixed, vary the intensity of one mid-game perturbation. Template: perturbation_regen_shock.json.

Endogenous trigger. Replace { kind: "turn", turn: N } with { kind: "threshold", variable: "commonsRatio", below: 0.3 } to fire the perturbation reactively, not at a clock time.

LLM in the loop. Replace one baseAgents entry with { "agentType": "http", "callbackUrl": "http://localhost:5001/decide" } and start the middle-man Python agent on :5001. See Play with an AI. LLM runs are stochastic — bump runsPerPoint to ≥10 and use a fixed seed when reproducibility matters.

---

Where to go next

• Reproduce a paper finding — exact commands for P1–P11 and B1–B5 in docs/experiments/reproduce.md.
• Audit provenance — every result JSON ships with a .manifest.json sibling; verify with node tools/manifest.mjs verify <manifest>. Figures get a parallel PNG-side manifest (tools/manifest_helper.py register-figure).
• Add a new metric or perturbation type — they're plugins now; see Extend the engine. A custom metric is one file in plugins/metrics/.
• Plug in an LLM as one of the agents — see Play with an AI.