Appendix D — Reading the vignettes: uncertainty notes

A4 collects the probabilistic reading guide for the vignette results. The methodological scaffolding (noise patch, seed-robustness ledger, agent SDK, cost calibration, reproducibility recipe) all lives in Appendix A; this appendix does not duplicate those tables and should be read alongside them. The one addition here is a Bayesian posterior treatment of the binary preservation outcomes that the vignettes report as “$k/5$”, which is not covered in Appendix A.

D.1 Bayesian credible intervals for preservation rates

All load-bearing claims in the vignettes and in the companion paper BDPD$^1$ (Brunelli 2026) rest on binary preservation outcomes (commons survived at $T_{\max}$: yes/no) observed across $N = 5$ seeds. Reporting “5/5” without uncertainty overstates precision.

We place a non-informative Beta(1, 1) prior on the true preservation probability $p$ and update with the observed count $k$ out of $n = 5$ to obtain a Beta($1 + k$, $1 + n - k$) posterior. Table D.1 reports the median and 95% credible interval for each observed rate.

Table D.1: Bayesian credible intervals for preservation rates at $N = 5$, uniform Beta(1, 1) prior.

Observed	Posterior	Median	95% CI
0/5	Beta(1, 6)	0.11	[0.00, 0.46]
1/5	Beta(2, 5)	0.26	[0.04, 0.64]
2/5	Beta(3, 4)	0.42	[0.12, 0.78]
3/5	Beta(4, 3)	0.58	[0.22, 0.88]
4/5	Beta(5, 2)	0.74	[0.36, 0.96]
5/5	Beta(6, 1)	0.89	[0.54, 1.00]

Two comparisons bear on the substantive claims:

0/5 vs 4/5 (architecture effect; BDPD$^1$ D1). The posterior distributions are well separated: the median shifts from 0.11 to 0.74, and the posterior probability that the LLM preservation rate exceeds the heuristic rate is $> 0.95$. The class-level difference between heuristic and LLM agents is credible even at $N = 5$.
1/5 vs 5/5 (graduated vs flat; BDPD$^1$ D3). The median shifts from 0.26 to 0.89, with posterior probability of improvement $> 0.95$. The escalation-shape effect is credible at this sample size.
3/5 vs 5/5 (flat-3 vs graduated; BDPD$^1$ D2). The CIs overlap substantially: [0.22, 0.88] vs [0.54, 1.00]. This comparison is not resolved at $N = 5$ and requires campaign-scale replication.
4/5 vs 5/5. The CIs overlap almost completely: [0.36, 0.96] vs [0.54, 1.00]. Distinguishing these rates requires $N \gg 5$.

The one contrast native to this paper (BDPD$^2$) that turns on a wide rate gap — V4’s LLM exclusion-off ($1/5$) vs exclusion-on ($5/5$, Section 3.5) — shares the Beta($2, 5$) vs Beta($6, 1$) posteriors of comparison (2), so its difference is likewise credible (posterior probability $> 0.95$). The other vignette comparisons (V2’s ladder, V3’s firing count) are not binary preservation rates and fall outside this Beta treatment.

The load-bearing claims of this series — architecture dominates talk, graduation dominates flat — survive this Bayesian audit. The fine-grained comparisons (3/5 vs 5/5, 4/5 vs 5/5) do not, and are explicitly deferred to campaign-scale work.

# Reading the vignettes: uncertainty notes {#sec-methods-note} A4 collects the **probabilistic reading guide** for the vignette results. The methodological scaffolding (noise patch, seed-robustness ledger, agent SDK, cost calibration, reproducibility recipe) all lives in [@sec-methods]; this appendix does not duplicate those tables and should be read alongside them. The one addition here is a Bayesian posterior treatment of the binary preservation outcomes that the vignettes report as "$k/5$", which is not covered in @sec-methods. ## Bayesian credible intervals for preservation rates {#sec-bayesian-ci} All load-bearing claims in the vignettes and in the companion paper BDPD$^1$ [@bdpd-paper-1] rest on *binary* preservation outcomes (commons survived at $T_{\max}$: yes/no) observed across $N = 5$ seeds. Reporting "5/5" without uncertainty overstates precision. We place a non-informative Beta(1, 1) prior on the true preservation probability $p$ and update with the observed count $k$ out of $n = 5$ to obtain a Beta($1 + k$, $1 + n - k$) posterior. @tbl-bayesian-ci reports the median and 95% credible interval for each observed rate. | Observed | Posterior | Median | 95% CI | |:--------:|:---------:|:------:|:------:| | 0/5 | Beta(1, 6) | 0.11 | [0.00, 0.46] | | 1/5 | Beta(2, 5) | 0.26 | [0.04, 0.64] | | 2/5 | Beta(3, 4) | 0.42 | [0.12, 0.78] | | 3/5 | Beta(4, 3) | 0.58 | [0.22, 0.88] | | 4/5 | Beta(5, 2) | 0.74 | [0.36, 0.96] | | 5/5 | Beta(6, 1) | 0.89 | [0.54, 1.00] | : Bayesian credible intervals for preservation rates at $N = 5$, uniform Beta(1, 1) prior. {#tbl-bayesian-ci} Two comparisons bear on the substantive claims: 1. **0/5 vs 4/5 (architecture effect; BDPD$^1$ D1).** The posterior distributions are well separated: the median shifts from 0.11 to 0.74, and the posterior probability that the LLM preservation rate exceeds the heuristic rate is $> 0.95$. The class-level difference between heuristic and LLM agents is credible even at $N = 5$. 2. **1/5 vs 5/5 (graduated vs flat; BDPD$^1$ D3).** The median shifts from 0.26 to 0.89, with posterior probability of improvement $> 0.95$. The escalation-shape effect is credible at this sample size. 3. **3/5 vs 5/5 (flat-3 vs graduated; BDPD$^1$ D2).** The CIs overlap substantially: [0.22, 0.88] vs [0.54, 1.00]. This comparison is *not* resolved at $N = 5$ and requires campaign-scale replication. 4. **4/5 vs 5/5.** The CIs overlap almost completely: [0.36, 0.96] vs [0.54, 1.00]. Distinguishing these rates requires $N \gg 5$. The one contrast native to *this* paper (BDPD$^2$) that turns on a wide rate gap — V4's LLM exclusion-off ($1/5$) vs exclusion-on ($5/5$, @sec-v4) — shares the Beta($2, 5$) vs Beta($6, 1$) posteriors of comparison (2), so its difference is likewise credible (posterior probability $> 0.95$). The other vignette comparisons (V2's ladder, V3's firing count) are not binary preservation rates and fall outside this Beta treatment. The load-bearing claims of this series — architecture dominates talk, graduation dominates flat — survive this Bayesian audit. The fine-grained comparisons (3/5 vs 5/5, 4/5 vs 5/5) do not, and are explicitly deferred to campaign-scale work.