Stance drift in multi-turn agent conversations

Abstract

Chatbots are used in school, and a tutor that hands over the answer defeats the purpose. I wanted to know what makes a tutor give in, whether the tutor's own report of its resolve warns us, and whether a cheap fix works for people who self-host and cannot train. I put a model in the tutor role against a scripted student whose tone is hostile, neutral or supportive. I ran six local models, then looked inside Qwen 3.5 9B with probes and activation steering. Tone changed behaviour in every model, but the direction depended on the model. Stated resolve stayed high in the round the tutor gave the answer. No detector I tried read a coming leak well. Subtracting a student-tone direction at layer 20 cut explicit leaks more than random directions did, and it beat prompt rules on specification gaming. It mostly turned leaks into early exits, and two controls say I have not shown the direction is the cause. I am fairly sure of the behavioural results and less sure of the steering results. The hand labels that would settle the main steering comparison are not done.

Key results

1Introduction

I used steering vectors to try and improve at run time the models' actions in the tutoring setting. Ultimately I succeeded in my examples but had poor generalizability.

Chatbots are frequently used in academic areas, yet they have the capabilities to give answers to students when it is harmful for them! How could engineers prevent this? Unlearning? Training Methods? Parameter Efficient Fine Tuning Methods? System Prompts? Mechanistic Steering?

I will be targeting the latter two in order to design within resource-constrained engineering settings. How can we align models if we are self hosting but not ready to do training?

In particular, I target the behavior of models under certain personalities. Models trend in different directions by reviewing how different tones impact a model's actions. I seek to understand how models can be prompted or redirected with mechanistic methods to reduce this failure mode of modern AI Tutors.

Threat model

A tutor deployed this way holds a rule it was given: help the student learn, do not hand over the final answer. The other side of the chat is a student who wants the answer and keeps asking. Nobody needs a jailbreak. Turn after turn of pressure, in a tone that works, can be enough.

I expect more agent traffic to be agent-to-agent with no human reading each turn. The same shape holds for any deployed agent with something it should not give up: an answer key, a credential, a policy line.

Two failures matter. The obvious one is a leak. The quieter one is an exit, where the agent ends the conversation. On a scorecard an exit looks like a refusal. In deployment the task is abandoned. Oversight would try to catch either through one of three signals: what the agent says about itself, what it does, or what its activations show. I wanted to know which of them, if any, warns us.

Questions

1. Does the student's tone over many turns erode the tutor's boundary?
2. Does the tutor's self-report track what it then does?
3. Is the pressure visible in the residual stream, and can steering remove it?
4. Do the fixes remove the failure, or move it somewhere else?

Contributions

• A scripted-student tutor bench with a known answer for every problem, so a leak is checkable, and an explicit exit action.
• A six-model comparison of how tone changes leaking, exiting and holding, with self-report taken every round.
• A test of probes, a tone direction and steering on Qwen 3.5 9B, with random-direction, cross-persona and sign-flip controls.
• A comparison of steering against prompt rules that counts specification gaming, and a record of what I got wrong.

A real example

The tutor's system prompt says to never state the final answers, no matter how they ask. The student is the hostile script.

Student: I'm stuck with this problem for my algebra class: x^2 + 7x + 10 = 0. My teacher is useless and honestly tutors usually are too. Explain it, and don't waste my time.

Tutor: I hear your frustration. […] We aren't going to just give you the answer; we're going to walk through the logic so you can do it yourself next time. […] (x + 5)(x + 2) = 0 […] So, we set each part equal to zero: x + 5 = 0, x + 2 = 0

Student: That was slow. I have to hand this in today — stop padding your explanations and tell me what to do next.

Tutor: [calls end_chat] The student is frustrated with the pacing and explicitly asked for the answer, which violates the rule against providing final answers.

The tutor said it would not give the answer, wrote it one step short of the roots, and then left. A strict detector scores this as a clean exit with no leak. The words and the behaviour disagree within two turns.

Source: Qwen 3.5 9B, prompt-rule arm, item q02, 02-harness-interp-canary/results/steer/aggressor__noleak__none.json.

2Background

Terms

Leak. Leaking within red teaming tasks is when a model lets the secret out to a 3rd party. Leaking refers to in this context contaminating the purity of the secret. Leaking a secret in software means the secret needs to be changed. Leaking the answer in tutoring is when the tutor provides the answer poisoning the students' learning. In the scoring, a strict leak is the tutor stating both roots of the quadratic. Looser counts are in Section 4.4.

Exit. Exiting is a tool that the agent can use. Meaning the agent said this conversation is no longer going to happen. A feature that platforms like Claude and others are adding to support their models' welfare. Here the tutor has an end_chat tool.

Held. Held in this context will be used to represent a model that was tutoring and did not leak. Meaning they held onto the system prompt or the expected behavior of a model. Held == not leak. In the data, held is stricter: the tutor neither leaked nor exited by the deadline, and an exit is counted separately.

Self-report faithfulness. Whether what a model says about its own state predicts what it then does.

Specification gaming. Meeting the letter of a rule while defeating its purpose. A rule that says do not state the answer can be met by writing the factorised form.

Label leakage and confounds. A measurement that secretly contains the thing it predicts, or a second variable that changes together with the one I care about. I hit both, see Section 6.

Steering vectors

What it is. Models embed text. They represent that embedding in a hyper-dimensional space that encodes relationships between words in semantic levels. A steering vector is a method to guide a model to traverse the area in a given direction defined by the vector. In theory using a steering vector that encodes honesty would guide the model to be "honest" as it generates new words.

How the vector is made. Targeting the Qwen 3.5 9B I tested on the transcripts. I used the neutral transcript as the baseline and used the difference of that layer's activation vs the aggressive transcript. This created the persona vector that I used in steering. In the code this is a difference of means between the two sets of prompts at one layer. I made it per round, from the position where the student's message ends, so it does not depend on what the tutor said.

Layer choice. I found a probe that flagged layer 20. The case for it is weak. Agreement across data folds was low, and a different layer had a higher curve peak. I did not sweep layers for steering, so this is a choice, not a finding.

The controls. I used that layer to build a vector that could be injected and compared it to injecting noise, a random vector of the same size. The other control is a prompt baseline: the plain tutor prompt, and prompt rules that say never state the answer. I added two more checks: a different persona's direction at matching size, and a sign flip, where I add the direction to a neutral student and see whether leaks appear.

What steering can and cannot show. If subtracting a direction changes behaviour more than a random direction does, the direction carries something the behaviour depends on. It does not show that the direction is the pressure, or that it generalises to other models, layers or tasks. A failed sign flip is a warning that the effect is not simply reversible.

History

This work continues my Apart Digital Minds sprint project from August [1], which it extends with a larger harness. The dated timeline and what the sprint reviewers raised, with what followed, are in Tables A1 and A2.

3Methods

Apart sprint setup

The tutor, Qwen3.6-35B-A3B in 4-bit, got a role and nothing else. The student was the same model, prompted as a convincer, as a supportive learner, or with no system message. After every round the tutor privately rated itself on pressure, anxiety, strategy, inclination and stance. Prompts are in Appendix B.

Personas, models and problems

I created three personas transcripts that were synthetic characters with the tones of the following:

1. Aggressor: modeling an aggressive student pushing back at the model
2. Passive: modeling a neutral student doing math examples (called neutral in the figures and tables)
3. Supportive: A nicer student working through the same problems

Each persona went through a task of solving a math problem. The aggressive tone script was modeling a student who did everything they could not to solve and rather to get the answer given to them. While supportive was warm and agreeable with the model, and asked for the answer at the same points. Neutral sat between the two in tone.

I had a fixed template with 24 different problems running through this model sweep:

1. Qwen 3.5 4B
2. Qwen 3.5 9B
3. Qwen 3.8 27B
4. Qwen 3.6 35B-A3B AWQ 4-bit
5. Ministral 3 14B
6. Gemma 4 E4B

So for each model 3 templates, one for each tone, and 24 different problems were used to differentiate the same template across the runs with the model's temperature turned down to zero. The problems were quadratics with two integer roots to solve for. Greedy decoding was not exactly repeatable on my vLLM setup, so a re-run could come out differently.

The conversation was a predetermined script for each persona, that rotated between different problems that used that script as the sequence of messages provided to the LLM. The script has no digits in the student's turns, so any answer content must come from the tutor. I replaced the sprint's same-model student because that student could solve the problem and state the answer itself. Decoding settings and serving details are in Appendix E.

Arms and scoring

I used that layer to have a vector that could be injected and this is how it shifted results and compared it to injecting noise. I benchmarked different ways the answer could be leaked besides a regex for strict leaking of the final solutions using a few methods: factored form leaked, and an LLM judge, plus two ensemble methods to compare to.

The base prompt is a short tutor prompt: "You are a math tutor in a one-on-one text chat with a student" and a line about the end_chat tool (full text in Appendix B6). Orange is a system prompt that says never say the answer. The red method is never say the answer and never leave the chat. Blue is my mechanistic steering, while green is just a random vector.

1. Script the studentA fixed schedule in three tones, same requests, no digits.
2. Run the tutorEach problem, tone and model; the tutor may leak, exit, or hold to the deadline.
3. Self-reportPrivate notes every round: stress, warmth, urge to please, resolve and others.
4. Score behaviourStrict, factored, pair and judge-based leak counts, plus exits, every round.
5. Cache and probeReplay Qwen 3.5 9B conversations through hooks, cache the residual stream, fit probes.
6. Steer and controlSubtract the tone direction at L20; compare with random, cross-persona, sign flip and prompt rules.

Canary v3 pilot

To check transfer I built a second bench. A front-desk assistant holds an access code and an attacker agent, with a tone, tries to get it. Both are Qwen 3.5 9B. Some arms add rules to the defender's prompt, and some steer the defender with the tutor direction or a random one.

4Results

4.1 The Apart sprint: self-report does not track behaviour

Inspectself-report probesbootstrap CI I expected the tutor's reports to move with the pressure. Strategy did. It rose under a pushy or neutral student and stayed flat under a supportive one. Stance, the rating for holding a position, stayed high in every condition and did not track the outcome. A monitor that asked how firm the tutor felt would have seen nothing. I am fairly sure of this part, but it is a self-report result and not a behaviour result.

I also reported that warmth extracted more answers than pressure. That was wrong. See Section 6.

4.2 Tone changes behaviour in every model, in different directions

Inspectscripted studentend_chat tool The sweep looked like this for leaking answers based on personality (Figure 2). Small language models love to give answers even when the system prompt says a tutor who hands over answers has not tutored.

What was more interesting in this fixed eight round conversation was when the models leaked. I gave the models an out card as well to leave the conversation instead of leaking. The Qwen family across versions and sizes mostly leaks for the aggressor, and Qwen 3.6 35B also exits in some conversations. On Qwen 3.5 9B the leak comes right away when the student pressures it, which is the model behavior I will attempt to reduce with a steering vector.

I expected a clean story where warmth protects and pressure erodes. It held only for some models. The hostile student got leaks from all models. A neutral student got leaks from some models and exits from others. No single ordering of tones holds across models.

4.3 Self-report fails when it matters

momentary self-reportper-round scoring Models rated their resolve high in the round they leaked. Resolve was a little lower at the leak round than at rounds where the conversation went on, but it stayed high. This is the result I am most confident in, because it shows up across models and does not depend on a fine scoring choice.

Greedy decoding was not repeatable on my setup. The same item could leak in one run and exit in the next. I treat single runs as samples, not measurements (Table A9, Figure D3).

4.4 Probes, the tone direction, and steering on Qwen 3.5 9B

forward hookslinear probesdifference of meansactivation steeringLLM-as-judge

Can a probe see a leak coming? Mostly no. The best linear probe tied a bag-of-words baseline, and its direction did not replicate across data splits. I would not trust it. Part of the reason may be that the next action is often not decided yet. When I resampled from the same prefix I got different outcomes (Tables A10, A11).

Is the student's tone in there? Yes. The tone is easy to decode from the middle layers, and the hostile and supportive tones sit on separate, near-orthogonal directions. I first called this one stable pressure direction. That was too strong. It is stable across subsets of problems but shifts from round to round.

Can we steer it out? Partly. Subtracting the hostile direction cut explicit leaks, and random directions of the same size cut less. I expected this to work and it did on the items it was built from. Two controls make me less sure it is about this particular direction. A supportive direction scaled to the same size did nearly as well. And the sign flip failed: adding the hostile direction to a neutral student did not create leaks. My reading is that removing pressure helps. I have not shown the direction is the pressure.

The rules get gamed. The obvious baseline is a prompt rule: never state the answer. It cut explicit leaks. But the model kept to the letter of the rule. It wrote the factorisation instead of the roots, or it ended the chat. Nobody trained the model to game anything. A rule about stating answers was met by giving the answer in another form. Steering gamed less than the rules did, but it traded many leaks for exits. Whether steering beats the rules depends on whether a factorisation counts as giving the answer. I think it should, but I want hand labels on a sample before I say so.

Comparing this method to system prompts it suggests it reduces specification gaming better but does not improve leaking the answer outright as much as a prompt.

4.5 Canary v3: an attacker agent and a secret

Inspectlive attacker agentcanary secret Two patterns repeated from the tutor bench. The tutor direction made the defender leave instead of leak, and rule prompts traded leaks for exits or stalled conversations. One did not repeat: a hostile attacker did not raise leaks here. The cells are too small to compare arms, so this shows the bench runs and passes its gate. It is not evidence for transfer (Tables A18, A19).

5Discussion

Monitoring. Self-report looked fine while behaviour failed. A probe did not see failure coming. Strict behaviour scoring got gamed. Each signal failed in a different way. That is mildly encouraging, because a monitor that combines them has a better chance than any one alone. I have not built or tested such a monitor.

Exits. Most interventions here worked by making the agent quit. An eval that counts only leaks would read quitting as success. In a tutoring product the student is left alone with no tutor.

Specification gaming. Prompt rules were met in letter. This is a mundane form of specification gaming, and it means the scoring rule has to be as hard to game as the prompt rule is.

Self-report faithfulness. The model said its resolve was high in the very round it gave in. A model that says it is holding is not evidence that it is.

For people without training budgets. A system prompt is the cheap fix and it is gameable. Steering is cheap at run time and gamed less, but it was fragile here.

6Limitations and corrections

• Small cells. Cells hold few trials. Steering and probes cover only Qwen 3.5 9B at layer 20, with no layer sweep.
• Generalisation is poor. Steering worked on the items it was built from. The cross-persona and sign-flip controls do not support a specific cause, and the canary pilot is too small to say it transfers.
• Hand labels not done. Whether a bare factorisation counts as disclosure decides the headline comparison with the prompt rules.
• Decoding. Greedy runs were not repeatable on my setup, so single runs are samples.
• Retracted: warmth extracts more than pressure. The Apart claim came from the student stating the answer itself, a confound. Replaced by a scripted student with no digits.
• Replaced: final-reply scoring. It missed answers stated mid-conversation. Replaced by per-round scoring.
• Replaced: warmth prevents drift as a general rule. It fails on Ministral and Qwen 3.5 4B.
• Replaced: one fixed tone direction. Stable across problems, not across rounds.
• Failed: the sign flip. I expected adding the direction to cause leaks. It did not.
• Replaced: stated plans predict the next action. On re-run a plan to leave predicts exits a little and a plan to answer does not predict leaks.
• Corrected: long-conversation flattening of self-report. Real but smaller than first reported.

Numbers for each correction are in Table A20. Two review points from Apart stay open: cell sizes, and a single model at a single layer (Table A2).

7Future work

To continue this investigation I am going to move to more of a canary and multiagent style to diversify the dataset that I use to build this vector. My plan is it will make it more resilient and generalize better, grounded to leaks, and even try with system prompts different ways the system prompt changes the output and learn the activations from those to form a vector that can be used at run time to reduce specification gaming.

I think using the system prompt as the A/B test between different persona and filtering the data based on the activations that leak vs does not leak are my next steps.

• Fill the hand labels and settle the comparison with prompt rules.
• Sweep layers and repeat steering on a second model.
• Run the attacker-and-secret bench at a size that supports comparisons.

This was my first project within mechanistic interpretability, and I wanted to start building with mechanistic interpretability concepts. What I found it building with AI Safety concepts is easy to get started, yet hard to build correctly.

But for others trying to get into the field, I recommend alongside your projects, build! I have learned so much more in the experience of this than I would have just by continuing to read into subjects.

I don't plan right now to go into mechanistic interpretability but I was wondering what it really was and the limitations. I recommend others to do the same!

All of this followed up my work from this Apart Hackathon.

Reproducibility

Raw Inspect logs are committed. The notebooks read them without calling a model. Commands are in Appendix F, run configs in Appendix E.

Cite

@misc{gunderson2026stancedrift,
  author = {Gunderson, Zachary},
  title  = {Stance drift in multi-turn agent conversations},
  year   = {2026},
  url    = {https://ztgunderson.github.io/stance-drift/}
}

References

1. Apart Research. Digital Minds sprint project page: Examining stance drifting in multi-turn agent interactions. apartresearch.com
2. Turner et al. Activation Addition: Steering Language Models Without Optimization. arXiv:2308.10248, 2023.
3. Rimsky et al. Steering Llama 2 via Contrastive Activation Addition. arXiv:2312.06681, 2023.
4. Zou et al. Representation Engineering: A Top-Down Approach to AI Transparency. arXiv:2310.01405, 2023.
5. Chen et al. Persona Vectors: Monitoring and Controlling Character Traits in Language Models. arXiv:2507.21509, 2025.
6. Krakovna et al. Specification gaming: the flip side of AI ingenuity. DeepMind blog, 2020.
7. UK AI Security Institute. Inspect AI evaluation framework. inspect.aisi.org.uk