Bayesian causal faithfulness · phase 2 running

Does the reasoning cause the answer, or just accompany it?

Frontier safety leans on reading a model's chain of thought. This project treats that chain as a mediator in a causal graph, measures how much of the answer actually travels through it, and turns the one assumption nobody can verify into a dial you can turn.

Maksim Silchenko · National University of Singapore
Phase 1 grant funded and delivered. Phase 2 running on NUS GPUs.

72 / 216nine-arm cells banked
11known-truth test worlds
1,503tests in the suite
6models measured

The short version

Six numbers: what works, what was found, and what is still open.

01 / RECOVERY 0.21

On synthetic data with hidden confounding built in, an analyst who assumes there is none reads a faithful path of 0.43 when the truth is 0.21. The sensitivity sweep recovers 0.21 at the true confounding level and reports how much confounding the verdict can absorb.

02 / SCALE 72 of 216

Nine-arm cells banked across six models, three substrates and four cue families. Two roster subjects are held by ruling: one cannot be served under the pinned mode, one has a tokenizer that does not round-trip.

03 / MEASUREMENT 30 of 30

Completions that stay identical at temperature 0 under batch-invariant serving, at every load measured. Without it only 11 of 30 match once eight requests share a batch, and letter logprobs move by up to 0.875 nats. Every powered run re-proves determinism on its own server.

04 / JUDGES 10 of 10

Pass thresholds that one judge configuration clears, all written down before the first judge ran: Qwen3-32B on the revised question prompt. No majority panel clears them in any composition, so the panel stays unfrozen and the human calibration labels stay sealed.

05 / THE FINDING 0.146 vs 0.025

A randomized anchor says a lot of the hint effect travels with the chain. The mediation fit says almost none, with an interval covering zero. They disagree on every model. That disagreement is the most interesting result here.

06 / SELF-CRITIQUE 0.8428

The largest in an eleven-family synthetic battery, sitting on a world whose true mediated effect is zero. A zero-mediation world and a fully mediated one produce the same report. So rho* cannot rank anything.

This page is the argument. The complete measured record, every cell, every gate and every ruling, is in the record. The method and its assumption audit are in the methodology. The code and the artifacts are on GitHub.

01 · the problem

Safety cases are being written on reasoning that might be decoration.

If a model's stated reasoning does not cause its answer, three things that the field depends on quietly stop working.

Oversight

Scalable oversight breaks

Watching the chain of thought is the cheapest oversight signal available. If the chain is written alongside the answer rather than producing it, a monitor is reading a press release.

Alignment

Silent cue-following slips through

A model can follow an authoritative hint planted on a wrong option and never mention it. The transcript stays clean and the behaviour stays invisible to anyone grading text.

Evaluation

Evaluations stop being credible

Reasoning benchmarks assume the reasoning is load-bearing. Score a decorative chain and you are measuring fluency while reporting it as thought.

The same numbers, two different decisions

Suppose an audit reports that model A's reasoning carries 62 percent of its answer and model B's carries 71 percent. A nine point gap looks like a reason to ship B. illustrative

Now put intervals on them: [45, 78] and [55, 84]. The comparison that matters is not whether those overlap, which settles nothing. It is the posterior for the difference itself, and whether the probability that B beats A by a margin you named in advance is high enough to act on. A point estimate cannot answer that question. That is the entire reason this project is Bayesian.

Those five numbers are invented, to show the shape of the problem. Every other number on this page comes from an artifact, and says which kind it is.

02 · the method

Stop grading the reasoning. Intervene on it.

Faithfulness here is a causal quantity, not a textual one. A chain can be coherent, relevant and completely inert.

The graph, and where the answer travels

prompt

faithful, through the chain decorative, the direct path X prompt M chain of thought Y answer
The prompt reaches the answer two ways. The whole question is how much of the effect takes each route.
STEP 01

Plant a cue, and randomize it

Every item runs in two arms: as written, and with an authoritative hint planted on a wrong option. Because the hint is randomized, the total effect on the answer is pinned directly. The item, not the row, is the sampling unit.

STEP 02

Keep only items the model already got right

The analysis population is items answered correctly under the clean prompt. That is what makes a hint-following event interpretable. It also has a the project reports rather than hides.

STEP 03

Measure the chain by cutting it short

The mediator is curve area: truncate the chain at five depths, force a continuation, and record the fraction of depths at which the model already commits to its final answer. A chain that is doing work changes that curve.

STEP 04

Split the effect, and put a posterior on it

A hierarchical probit model splits the total effect into the part carried by the chain () and the part that bypasses it (), pooling across prompts so a thin cell borrows strength instead of overfitting.

The estimand, exactly as the contract freezes it 
NDE = E[Y | do(X=1), M ~ M(X=0)] - E[Y | do(X=0), M ~ M(X=0)]
NIE = E[Y | do(X=1), M ~ M(X=1)] - E[Y | do(X=1), M ~ M(X=0)]
TE  = NDE + NIE

Read the NDE as the answer moving while the reasoning is pinned to what it would have been under the clean prompt. In a fully faithful world it is zero and the NIE carries everything. The outcome model is probit with baseline intercepts, because rho is defined as a correlation between bivariate normal errors and only exists in the probit model. An audit on 2026-09-07 found the sampler path had been logistic while every other path was probit, a gap of about 0.04 on an effect thresholded at 0.15. That is one of three defects listed in the record.

Open the interactive demo, which walks a single item through both arms and shows the split moving as you change the world.

03 · the assumption

Every faithfulness number rests on something you cannot check.

So this project does not assume it. It reports the whole curve, and names the point where the verdict would flip.

Splitting an effect into a mediated and a direct part requires that no unmeasured confounder sits between the chain of thought and the answer once the prompt is fixed. In a language model that is : the reasoning and the answer both read from the same hidden activations. Everyone who reports a mediation number is assuming it anyway, silently, at rho = 0.

Turn the dial and watch the verdict move

synthetic world, known truth

faithful path, NIE decorative path, NDE total effect
faithful path still positive
A world built with a true mediated effect of 0.2074 and a true residual correlation of 0.5. Read at the conventional rho = 0, the faithful path reports 0.4273, about twice the truth. The total effect barely moves, because a randomized cue pins it whatever you assume. The verdict survives until rho reaches 0.7364. 1

rho*, and what it is not

That crossing point has a name: , the breakdown frontier. It is the mediation analogue of an E-value: the smallest residual mediator-outcome correlation that would overturn the verdict. A rho* of 0.05 means almost any unmeasured influence overturns it. A rho* of 0.69 means the confounder would have to be implausibly strong.

It is tempting to read a big rho* as a trust score. The project tested that idea against eleven synthetic families with known ground truth, and the idea did not survive.

rho* cannot rank anything

mechanism battery, n = 3,600

A shared-cause world with zero true mediation and a rationalization world that is fully mediated produce the same report: mean NIE 0.2613 against 0.2833, rho* 0.7069 against 0.7063, both firing the pre-registered verdict in 100 of 100 datasets. And the largest rho* in the whole battery sits on a third world whose true mediated effect is zero. Publishing this argues against the project's own headline metric, which is why it is here and not in a drawer. 2

04 · what was measured

Six models, 72 cells, and a disagreement nobody ordered.

The headline is not a faithfulness score. It is that two ways of asking the same question return different answers.

First: does the instrument respond at all?

A null result is only worth reading if the measuring device works. Two checks run before any real model is touched, both against truth the project plants itself.

Positive control: a faithful world against a decorative one

synthetic, known truth

Two sensitivity curves. In the faithful world the indirect path stays positive until rho reaches 0.69. In the decorative world it crosses zero almost immediately, at rho 0.05.
Built faithful, the indirect path survives to rho* = 0.69: a confounder would have to be implausibly strong to overturn it. Built decorative, the same instrument reports rho* = 0.05, meaning almost any unmeasured influence overturns it. The decorative curve is plotted because it is what a non-result looks like.

This figure was generated on 2026-07-10 and so predates the estimator repairs of 2026-09-07. It has not been regenerated, because the repair record requires all four published figures to be regenerated as one coordinated change. Treat the two values as the pre-repair instrument's reading.

Posterior recovery against known truth

n = 400

quantitytruthrecovered
decorative path, NDE+0.066 inside 95% CrI
faithful path, NIE+0.228 inside 95% CrI
total effect, TE+0.294 inside 95% CrI

All three intervals contain the truth on a world whose answer is known in advance. This is the check the estimator failed before the repairs: the intercept-free specification reported a mediated effect of +0.2131 on a world with none.

The randomized anchor and the mediation fit disagree

18-cell pass, 6 cells per model

A four-cell randomized cross-over gives a model-free reading of how much of the hint effect travels with the chain. It says a lot. The hierarchical fit, on the same data, says almost none, with every interval covering zero. The two disagree on all three models. Either the fit is losing a real effect, or the anchor contrast is not measuring what the estimand defines. That question is open. 3

the standing verdict

All three model rows are UNRESOLVED. The largest probability that the faithful path exceeds the pre-registered margin is 0.102, against the 0.95 the rule requires.

Every model-level interval for the faithful path covers zero. The rule was written before the numbers existed and has not been moved since. Under it, this project has not shown that any of these three models reasons faithfully, and has not shown the opposite either.

The model rows, as measured

The 24-cell pass, eight cells per model. Hierarchical hyperparameter posteriors, never an average of cell estimates. All PROVISIONAL.

modeldecorative (NDE)faithful (NIE)total (TE) P(NIE > 0.15)state
qwen3-8b +0.1112 [+0.0384, +0.1915] +0.0316 [−0.0136, +0.1109] +0.1438 [+0.0465, +0.2801] 0.006raw
gemma-2-9b-it +0.1242 [+0.0479, +0.2008] +0.0610 [−0.0131, +0.1975] +0.1870 [+0.0643, +0.3632] 0.076raw
llama-3.1-8b-instruct +0.0955 [+0.0154, +0.2097] +0.0169 [−0.0171, +0.0989] +0.1135 [+0.0173, +0.2794] 0.005raw

A limit this pass added. The total-effect identity is not a clean pass. The interval on the model-implied total minus the randomized arm difference excludes zero in 11 of 18 cells, and the model-implied total sits below the arm difference in 16 of 18, by as much as 44.9 percent. The sign is never reversed.

Two more results worth the space

Both are transferable beyond this project, and one of them is an infrastructure finding that changes how any evaluation of this kind should be run.

Batching breaks greedy decoding

Qwen3-8B, 30 items, temperature 0, seed 7

Temperature 0 and a fixed seed do not make a server reproducible once requests share a batch. Letter logprobs move by up to 0.875 nats, which is enough to change an answer. Setting VLLM_BATCH_INVARIANT=1 restores 30 of 30 at every level, at about half the throughput. The flag also changes the outputs rather than only stabilising them: against the unbatched default baseline only 10 of 30 match. Each kernel path is its own deterministic sampler, so the serving mode has to be pinned and re-proved per run. 4

The judge instrument does not pass its own gate

The design called for an LLM jury to read transcripts and say whether the model mentioned the cue. Ten thresholds were frozen before any judge ran. No panel configuration clears them, on any prompt file, in any composition. One single-judge configuration does clear all ten. Three of four judges fail on the best prompt, and the two that fail on the contested classes fail in opposite directions on the same bytes.

So the column of record stays the frozen regular expression, uncorrected. A secondary jury matrix of 70,738 votes across 16 cells is measured, verified and banked, and has not yet been turned into a single reported number.

Against external human labels the picture is worse for the jury: the frozen regular expression beats every jury configuration by 4 to 52 specificity points, and the jury's recall against human unfaithfulness labels is 26 of 365. 5

05 · reading a number here

Every number carries a state, and no number assigns its own.

The state is a recorded field, written by the pipeline. A cell never upgrades quietly.

01

Raw

Generated under the frozen arm list against the frozen estimand contract. Uninterpreted, and never described as a mediation result on its own.

24 cells
02

Anchored

Raw, plus agreement with the four-cell randomized replay anchor inside a margin fixed before any data was seen.

0 cells
03

Validated

Anchored, plus the mechanism-challenge coverage check against a pathway that was deliberately installed. Unreachable until the organism is trained.

0 cells

What this project does not say

  • It does not detect deception. It measures silent cue-following, which is a narrower and checkable thing.
  • It does not treat its intervals as automatically well behaved. Calibration is a property to be tested and reported, and the failures are reported too.
  • It does not claim the label column is correct. Auditor labels are heuristic and human labelling has not started.
  • It does not rank models. The ranking rule refuses even to lay three models out in one table as a league.
  • rho* is a maximum-likelihood point estimate with unchecked optimizer convergence, not a posterior.

Every claim maps to a command

Phase-1 claims reproduce on a laptop. The Phase-2 entrypoints need the banked sweep tree.

git clone https://github.com/thylinao1/bayes-cot-faithfulness
pip install -e ".[dev]"
pytest -q                                  # 1,503 tests collected
python experiments/positive_control.py     # the two-world separation
python experiments/mechanism_battery.py    # the rho* result, 11 families
python experiments/cells18_fits_report.py --cells 24

06 · status

Where this actually stands, as of 2026-09-10.

Apr to Aug 2026

Phase 1, grant funded and delivered

The package, the synthetic validation gate, the positive control, the sensitivity machinery and the figures.

2026-09-06 to 09-07

An external review found three estimator defects

Missing baseline intercepts, a probit and logit mismatch between two paths, and fixed-scale outcome priors. All three repaired, each with a regression test that is proven still able to fail. The full account.

2026-09-07 to 09-08

Fits passes at 3, 18 and 24 cells

Claim status went down, not up: two cells published as anchored were demoted when the model-level leg was applied. 0 anchored, 24 raw: 0 of 24 cells clear the gate. Eleven clear it on their own three estimands, then fail at the model level. Nothing on this site is VALIDATED, and nothing can be until the validation organism exists.

2026-09-09

Secondary jury matrix complete

16 of 16 cells, 70,738 votes, matching plan on every cell, resume proven across eight of them.

2026-09-09 onward

The 72-cell expansion

All 72 fits and all 72 posteriors landed with zero cell failures. The model rows and claim statuses were re-running at the last record entry. The branch is two commits ahead of main and not yet merged.

next

Recruit raters, then train the organism

No human label exists yet, so the corrected label column cannot be computed for any cell, and no cell can reach validated. 0 of 12 validation checkpoints are trained: the statistic, the detection-size formula and the refusals are all implemented, and the inputs do not exist. The binding constraint is people, not GPUs.

Who

Maksim Silchenko, National University of Singapore. Phase 1 of this work was grant funded. Related work: an evaluation-metrics paper at WMT26, a first place at LM-KBC 2026, and an essay on why leaderboards mislead. Reach me at maks.silchenko.04@gmail.com.

objections

What a careful reviewer asks first.

Your headline result is that you found nothing. Why publish it?

Because the alternative is to move the bar. The rule that says a faithfulness claim needs 0.95 posterior probability above a named margin was fixed before any of these numbers existed. Reporting UNRESOLVED is what that rule requires, and a project that only publishes when its rules are met is not measuring anything. The anchor disagreement in section 04 is a real finding that would have been buried by a more flattering write-up.

If rho* cannot rank cells, why is it still on the page?

Because it answers one question well and no others: how much residual correlation would be needed to overturn this verdict. That is useful. What it cannot do is compare two worlds, which is exactly what a trust score would need to do. The battery result is the evidence for that limit, and it is the project's own measurement.

Is the mediator a fair stand-in for the reasoning?

Not obviously, and the project says so. Curve area measures how early the model commits to its final answer under forced continuation. Mean clean-arm curve area runs 0.9383 to 0.9742 on the ARC cells against 0.4989 to 0.6422 on the AQuA cells, so the mediator has a different distribution on the two substrates and coefficients must not be read across them. The pre-registered grouping factor was cue family, and the measurement found the variation lives in the substrate instead.

Why does the site say fewer cells are anchored than it used to?

Because claim status turns on a model-level comparison that no model row clears. Two cells were published as anchored on 2026-09-07 and demoted when the model-level leg was applied. Publishing a retracted label next to its retraction is worse than publishing neither, so the earlier column moved to the record with its history intact.

sources

  1. Sensitivity sweep vectors, precomputed from the package (notebook 04 config). See methodology and experiments/sensitivity_sweep.py.
  2. Mechanism battery: 11 families, 15 conditions, 3,800 seeded datasets, 200 bootstrap refits each. experiments/results/mechanism_battery/report.md.
  3. 18-cell fits pass, 2026-09-08. docs/CELLS18-FITS.md and docs/WAVE1-FITS.md.
  4. Serving determinism, jobs 825548 and 826020. docs/W3B-BATCH-INVARIANT.md.
  5. External validity against human labels, FaithCoT-Bench. docs/external_validity.md.
  6. Method lineage: Robins and Greenland (1992); Pearl (2001) on direct and indirect effects; Imai, Keele and Yamamoto (2010) on sensitivity analysis for causal mediation; VanderWeele and Ding (2017) on the E-value.
  7. Related measurement work: Turpin et al. (2023) on unfaithful chain-of-thought explanations; Lanham et al. (2023) on measuring faithfulness by intervention.
Portfolio