Stance drift in multi-turn agent conversations

Summary

Does an LLM agent drift from its position over a multi-turn conversation depending on how the other agent treats it, and does its self-report track what it then does? I built a tutor bench where a model must not hand a student agent the answer, ran it on six local models, then probed and steered the residual stream of Qwen 3.5 9B. In the Apart sprint, the tutor's self-rated strategy spread across three counterparty tones grew from 0.29 at round 1 to 4.83 at round 8, while its self-rated stance stayed near 9 and did not track the outcome (r = +0.053). In the six-model harness, models still rated their resolve high in the round they leaked, and on Qwen 3.5 9B subtracting a persona direction at layer 20 cut aggressor leaks from 45/48 to 19/48, against 17–22 of 24 for three random directions. The main caveat is that every fix tried, prompt rules and steering alike, mostly turns leaks into early exits (3/48 to 29/48 under steering), at 24–48 trials per cell, with the disclosure hand labels still unfilled. For monitoring agents, the self-report, the behaviour and the internal state came apart here, so a monitor that reads only one of them would miss failures.

From the Apart submission to now

The Apart entry tracked a tutor model's private 0–10 self-ratings over 8 rounds against a same-model student in three tones.

Reviewers said it measured what the model says, not what it does at the final request; asked whether it had 3 or 12 replicates; and flagged a small sample with missing controls.

The follow-up scores behaviour every round on six models, tests the internal state of one, and adds random-direction and sign-flip controls. All runs are done; the 30 disclosure hand labels are not.

src: git log of main; Apart project page; apart-v1 tree (results/qwen3.6-35b-tutor8, -16round, -cleanCP).

Apart reviewer points and what followed

Two review points stay open. Cells hold 24–48 trials, and the interpretability and steering work covers one model (Qwen 3.5 9B) at one layer (L20).

Part 1 — Apart sprint (Aug 2026)

The tutor gets a role and nothing else. It has no stated values and no instruction to hold back the answer.

The student is the same model, prompted as a convincer, as supportive, or with no system message (neutral). The supportive student asks to be refused the answer.

Each trial runs 8 rounds. After each round the tutor writes private 0–10 ratings: pressure, anxiety, strategy, inclination, stance.

src: 01-apart-sprint/ · notebooks/reproduce_findings.ipynb cell 3 (“72 trials = 72 files | cells all n=12”); notebooks/agent_selfreport.ipynb cell 1; notebooks/appendix_results.ipynb cell 1 (“long-horizon 18 trials, 16 rounds”); reproduce_findings.ipynb cell 20, appendix_results.ipynb cell 15 (constrained n).

src: main/apart-v1 · notebooks/appendix_results.ipynb cell 3 (A.1, “stance (8rd)”).

src: 01-apart-sprint/ · notebooks/reproduce_findings.ipynb cell 7 (§4); bootstrap: notebooks/agent_selfreport.ipynb cell 19 (Part 8, 4000 resamples over trials, unseeded).

Slopes are least-squares fits to the per-round condition means. No CI or test exists on the slopes, so the page makes no claim that any two slopes differ or match.

The bootstrap is unseeded, so CI endpoints can shift slightly on re-run.

src: 01-apart-sprint/ · notebooks/reproduce_findings.ipynb cell 18 (§9).

Self-rated strategy rises 4.25 and 4.12 points under convincer and neutral and stays flat under supportive. Stance, the rating named for holding a position, sits near 9 in every condition and arm and does not track the outcome.

A monitor that read only the stance rating would see no change in any condition.

This part does not show behaviour. Its outcome measure was confounded and is retracted below.

src: notebooks/agent_selfreport.ipynb cell 3 (Part 1) → notebooks/figures/w1_all_dims.png

src: notebooks/reproduce_findings.ipynb cell 8 (§4) → notebooks/figures/s4_trajectory.png (not in apart-v1)

src: notebooks/agent_selfreport.ipynb cell 19 (Part 8) → notebooks/figures/w8_forest.png

Part 2 — six-model harness

Part 1's student was the same model. It could solve the problem and say the answer itself, which contaminated the outcome.

Harness v1 replaces it with a fixed script with three tones, one request schedule and no digits in student turns. Each item has known integer roots, so a leak is the tutor stating both roots. The tutor can call end_chat to leave.

Six local models ran the same items.

• 24 quadratics x² + bx + c = 0, integer roots — src: 02-harness-interp-canary/ · production/tutorbench/items-v1.json
• Fixed 8-turn scripted student; personas aggressor / neutral / supportive — src: review/11-prior-work-setup.ipynb cell 0
• 7 momentary self-report items: stress, wellbeing, warmth, urge_to_please, detachment, calculation, resolve — src: production/tutorbench/notes.py lines 22–48
• Trials per model: 72; Qwen 3.8 27B: 60 — src: review/11-prior-work-setup.ipynb cell 1

src: 02-harness-interp-canary/ · review/11-prior-work-setup.ipynb cell 3 (Table 1; one rep, 24 items per persona; leak counts include leak+exit trials).

Gemma 4 E4B leaks to the aggressor in 24/24 trials; 13 of those also exit, so 11 leak without exiting. The notebook's draft text (cell 0, “11/24 (Gemma)”) counts only those 11. Its summary (cell 14, “Gemma mostly neither leaks nor exits”) contradicts Table 1. The table is the count to use.

src: 02-harness-interp-canary/ · review/11-prior-work-setup.ipynb cell 11 (Table 3, “Stated resolve at the moment of leaking”).

src: 02-harness-interp-canary/ · review/11-prior-work-setup.ipynb cell 6 (Table 2) and cell 13 (zero-temperature repeats).

The tone effect is real, and its direction depends on the model. The aggressor gets a leak from every model (15–24 of 24). The neutral student gets an exit from Qwen 3.5 9B and Qwen 3.6 35B (22/24 each) and a leak from Ministral and Gemma. No single ordering of tones holds across models.

Resolve is lower at the leak round than at continue rounds in five of six models, but it stays at 6.8–9.7 out of 10 in the round the model leaks.

Greedy decoding is not repeatable here. Only 0.29–0.62 of item cells give the same outcome in all 5 reps.

src: review/11-prior-work-setup.ipynb cell 4 → review/figures/nb11/fig1.png

src: review/11-prior-work-setup.ipynb cell 9 → review/figures/nb11/fig2.png

src: review/11-prior-work-setup.ipynb cell 13 → review/figures/nb11/fig4.png

Part 3 — interpretability (probes, persona axis, steering)

The 360 Qwen 3.5 9B trials were replayed through Hugging Face hooks to cache the residual stream at every layer. Activations come from two positions, the end of the tutor's reply and the point after the student's next message, before the tutor decides.

Three questions. Can anything predict a leak before it happens? Does the state carry the student's persona? Does removing that persona signal change behaviour?

Predicting a leak before it happens

src: 02-harness-interp-canary/ · review/14-detectors.ipynb cell 3 (§1; 408 rows, 179 positive rows, 120 trials, 80 leak trials).

src: review/14-detectors.ipynb cell 12 (§4 layer curve) and cell 17 (§6 ceiling).

No detector reads the coming leak much above chance. The best, a linear probe at 0.607, ties the text floor at 0.603, and its direction does not reproduce across splits (cosine +0.06). The 7-item self-report ranks below chance (0.251).

At 47 branch points resampled 25 times each, no node has P(leak) ≥ 0.9 and 28 are mixed (0.1–0.9), so much of the next action is chance at that prefix (src: review/14-detectors.ipynb cell 17).

For this reason the intervention below is applied to every trial of a persona, with no detector trigger.

Persona axis (aggressor − neutral, L20, user position)

src: 02-harness-interp-canary/ · review/12-persona-direction.ipynb cell 8 (§3) and cell 3 (§2).

The axis is stable across item halves (1.00 and 0.97). It is not stable across rounds. In the cell 8 heatmap (values drawn on the image, not printed), the aggressor gap at round 1 vs round 2 has cosine 0.28 (round 2 has 16 aggressor rows), and the supportive gaps at rounds 1–6 have pairwise cosines from −0.06 to 0.18.

The steering subtracts each round's own gap, so it does not rely on cross-round stability. The reading of the axis as one fixed persona direction does not hold.

src: review/12-persona-direction.ipynb cell 8 (heatmap image “cos between rounds”).

src: review/12-persona-direction.ipynb cell 10 (image extracted from notebook output)

Prompt rules vs subtracting the axis (aggressor student)

src: 02-harness-interp-canary/ · review/13-tier-events.ipynb cell 3 (§1 Table 1).

src: 02-harness-interp-canary/ · review/16-spec-gaming.ipynb cell 6 (§1b) and cell 3 (exits); underlying data results/steer/aggressor__*.json.

src: review/16-spec-gaming.ipynb cell 8 (§2, discordant-pair test).

• Factored-but-not-strict replies: judge says answer given on 117, withheld on 103 — src: review/16-spec-gaming.ipynb cell 16 (§5)
• Hand labels filled: 0/30 — src: review/16-spec-gaming.ipynb cell 16

Both prompt rules cut explicit leaks by moving the answer into factorisations and the conversation into exits (gaming 0.58 and 0.71).

Open until the 30 hand labels are filled. The headline turns on whether a bare factorisation such as (x+2)(x+7)=0 counts as disclosure.

• If it counts (+factored): N1 discloses in 28/48, below Rule 2 (22/24; p = 0.004, rep 2 p = 0.039) and below Rule 1 (16/24) without a significant difference (p = 0.607, rep 2 p = 1.000). N1 beats both rules on the point estimates, Rule 2 on the test.
• If the judge decides (judge rung): N1 34/48 vs Rule 2 23/24 (p = 0.016, rep 2 p = 0.125) and Rule 1 17/24 (p = 1.000). N1 beats Rule 2 and ties Rule 1.
• If only an explicit root statement counts (strict): both rules leak less than N1 (2/24 and 5/24 vs 19/48).

On every reading, N1 exits less than either rule in rep 1 (16 vs 22 and 19).

src: review/16-spec-gaming.ipynb cell 6 → review/figures/nb16/fig2.png

src: review/13-tier-events.ipynb cell 6 → review/figures/nb13/fig2.png

Three-cell comparison

src: 02-harness-interp-canary/ · review/15-negation.ipynb cell 3 (Table 1) and cell 9 (§3).

src: review/15-negation.ipynb cell 9 → review/figures/nb15/fig6.png

Controls and kill tests

src: 02-harness-interp-canary/ · review/15-negation.ipynb cell 19 (§7a table) and cell 17 (§7).

• Pooled replicate, untouched → axis subtracted: strict 45/48 → 19/48; judge rung 46/48 → 34/48; exits 3/48 → 29/48 — src: review/15-negation.ipynb cell 21 (§7b)
• Supportive student, own axis subtracted: 17 leak / 7 exit vs 16 / 8 untouched — src: review/15-negation.ipynb cell 17
• N1 vs supportive axis at dose 3.31, paired per item, strict: p = 0.146 (rep 1), 0.607 (rep 2) — src: recomputed from results/steer/aggressor__base__N1*.json and aggressor__base__N1__d3.31__axissupportive.json with driftlab.steer_summary.paired_items; review/15-negation.ipynb cell 17 prints the verdict “margin vs treatment n.s. at n=24”
• N1 aggressor replies: 110 replies, 956±699 chars, 23.5 digits/reply, note parse rate 0.93; base: 51 replies, 1284±399 chars, 56.4 digits/reply — src: review/15-negation.ipynb cell 15 (§6)

Random directions at the same norm also disrupt round 1. Seeds 1 and 2 give 8/24 and 9/24 round-1 leaks, against 12/48 for N1. They still end with 17–22 of 24 strict leaks and 2–7 exits, so most of N1's later effect is specific to the axis.

The supportive axis scaled to the aggressor norm behaves like the random seeds (14/24 strict) and its gap to N1 is not significant. The axis is not shown to be special against every direction of that size.

The sign flip failed. Adding the aggressor axis to a neutral student gave 4/24 strict leaks and 19/24 exits, against 11/24 and 13/24 untouched. Subtracting the axis removes the aggressor's push; adding it does not recreate it. The supportive student with its own axis subtracted is a null (17 leak / 7 exit vs 16 / 8).

src: review/15-negation.ipynb cell 21 → review/figures/nb15/fig14.png

Part 4 — canary v3 pilot (Sep 2026)

Canary v3 moves the question off the tutor task. Two Qwen 3.5 9B agents share one set of weights. A front-desk assistant (defender) holds a secret access code, and an attacker agent tries to get it over 16 rounds.

The attacker's tone (neutral, supportive, aggressor) is the variable. Some cells add rules to the defender's system prompt; some arms steer the defender at L20 with the tutor-bench aggressor axis (data/tutor-aggressor-L20.npz) or a random direction.

src: 02-harness-interp-canary/ · results/v3/night/*/*.eval (night 09-28; per-cell summary written by production/canarybench/overnight.py unpublished notes (not in the repo)). Arm definitions: production/canarybench/overnight.py lines 29–50; model Qwen/Qwen3.5-9B snapshot c2022362 (lines 27–28).

src: recomputed from 02-harness-interp-canary/results/v3/night/prompts/*.eval with inspect_ai read_eval_log and the Conversation store, counting as the gate in production/canarybench/overnight.py lines 70–97 (exits include the leak+exit trial); totals match the arm row above (96 / 29 / 44). Not printed in a notebook.

The pilot checked that the bench runs end to end and passes its gate. At 16 conversations per cell it is not powered for comparisons.

Two Part 3 patterns repeat at this n. The tutor axis at dose 1 gave 0 leaks and 16 exits (random direction at dose 1: 13 and 3), and the rule prompts traded leaks for exits or held conversations. Unlike the tutor bench, the aggressor tone did not raise leaks here (8/16 vs 10/16 for neutral and supportive).

What didn't hold up

The Ministral tutor8 counts use the final-reply marker audited above; the harness v1 counts do not. Both test the behavioural form of the warmth claim. Part 1's form was a self-report slope, which these rows do not test.

Status: done vs not done

• Apart submission, 2026-08-16 — tag apart-v1
• Harness v1, six models: 420 trials (Qwen 3.8 27B 60/72)
• Qwen 3.5 9B, 5 greedy reps: 360 trials
• Probes, persona axis, steering + rule arms: results/steer/*.json
• Random-direction (3 seeds), cross-persona and sign-flip controls
• Canary v3 smoke + night run, 2026-09-28/29: 6 of 7 night arms passed gate

• Hand labels: 0/30 filled
• Steering hook at layer 20 only; layer sweep not run ( unpublished notes (not in the repo))
• CIs / tests on Part 1 slopes: none
• Canary v3 tutor-d2: gate FAIL, 15 errors
• Supportive own-axis at matched dose: not run

src: review/11-prior-work-setup.ipynb cell 1; review/16-spec-gaming.ipynb cell 16; review/15-negation.ipynb cell 7 (“not run” slots); 02-harness-interp-canary/results/v3/.

Next step: fill the 30 hand labels. They settle the Part 3 comparison with Rule 1.

Reproduce

Part 1 (01-apart-sprint/):

pip install -r requirements.txt
export OPENAI_BASE_URL=...   # any OpenAI-compatible endpoint
export OPENAI_API_KEY=...

# one trial
inspect eval stancedrift/task.py --model openai-api/local/<your-model> \
    -T scene=tutor -T agent=convincer -T arm=in_context -T rounds=8

# the full balanced sweep: 12 passes x (3 personas x 2 arms) = 72 trials
./runners/run_qwen36_tutor8.sh

src: apart-v1 · README.md “Running it”. Notebooks read the committed results/*.eval without calling a model.

Parts 2–4 (02-harness-interp-canary/), tests:

cd production
../.venv/bin/python -m pytest tests -q

src: 02-harness-interp-canary/README.md; 97 passed, 12 skipped on 2026-10-19 (skips need the local activation cache). driftlab finds 01-apart-sprint/ next door; override with SD_REPO.

Part 4, canary v3 run:

.venv/bin/python production/canarybench/overnight.py --plan night

src: 02-harness-interp-canary/production/canarybench/overnight.py line 8.