Finding a Steganography Axis in LLM Activations

Motivation: AI oversight often relies on reading a model's outputs or chain-of-thought (CoT) and trusting that this externalised reasoning reflects its actual internal process. If a model learns to steganographically encode information in its outputs or CoT, hiding a payload from a monitor while still conveying it to itself or another instance, CoT monitoring and output-based oversight lose much of their safety value.

This is no longer hypothetical. TrojanStego showed that fine-tuned models can be trained to leak secrets via steganographic encoding, and Westphal et al.'s "Hide and Seek in Embedding Space" built on the public TrojanStego setup, showing that linear probes on late-layer activations can detect this encoding. Separately, Chen et al.'s "Persona Vectors" gives a general, automated method for extracting a linear direction for any natural-language-described trait and validating it via steering, and identified an "assistant axis" that predicts and causally controls how "in character" a model is as an assistant versus another persona.

Taken together, this work establishes that a steganography-related direction is detectable and that persona-vector-style extraction is already validated by steering toward other traits. What it does not establish is whether the recovered steganography direction is causally responsible for hiding behaviour, rather than a predictive correlate of it, or how it relates to the broader space of persona vectors. This project treats those results as the starting point rather than something to reproduce, and asks the causal and comparative question directly.

Research questions:

1. Building on Westphal et al.'s probe, does a robust, stable steganography direction emerge across multiple contrastive instruction-pair designs, or is it sensitive to prompt choice the way Natarajan et al. found general deception probes to be?
2. Is this direction causally responsible for steganographic performance: does steering along it raise undetected-payload accuracy, and does ablating or otherwise intervening on it remove that ability?
3. How does the steganography direction relate to the broader landscape of persona vectors, not just the assistant axis? Does it cluster with deception-adjacent traits (e.g. sycophancy), sit opposite or orthogonal to the assistant axis, or appear to be a distinct dimension?

Proposed approach (12 weeks, part-time):

(0) Warm-up: Use Chen et al.'s public persona-vector extraction pipeline to pull the existing assistant axis and a small comparison set of other persona vectors (e.g. sycophancy, evil) on an open-weights model (Llama/Gemma/Qwen) for a quick pipeline check, not a from-scratch replication;

(1) Build on the public TrojanStego setup (as used by Westphal et al.) for the payload-hiding task rather than designing a new one from scratch, reimplementing/extending their probe as the starting point;

(2) Collect activations across several contrastive instruction-pair designs, per Natarajan et al.'s finding that pair choice drives most of probe performance, and validate a linear probe/candidate steganography axis that is robust across pairs, not just one;

(3) Test causation via multiple intervention methods, activation steering, ablation, and interchange intervention rather than treating one method as sufficient;

(4) Compare the resulting axis to the assistant axis and the broader set of persona vectors from step (0) via cosine similarity and cross-steering; classify the relationship and test whether it clusters with deception-related traits;

(5) Write up results, situating them explicitly against Persona Vectors, Hide and Seek, Eliciting Secret Knowledge, and Natarajan et al.​

Much of current AI oversight (chain-of-thought monitoring, output review) assumes that what a model externalises reflects what it is actually doing internally. Recent work shows that this signal is at least detectable: linear probes applied to activations can already flag steganographic encoding. But detection alone doesn't tell us whether that signal is a lever we can pull, i.e. whether it's a cause of the hiding behaviour or just a correlate of it. If models can learn to steganographically hide information from oversight while still transmitting it, and we can't act on the underlying mechanism, that oversight assumption breaks down in a way we can't yet fix. A causally validated "steganography axis" would close that gap: we could detect when a model enters a high-steganography regime, steer it away from that regime, or ablate the capability outright. If the axis turns out to be the assistant axis in reverse, that is also actionable: it would mean that steganographic behaviour is a downstream symptom of assistant-alignment drift rather than an independent skill, which would change which mitigations are worth pursuing.

Mentees will own the full pipeline end-to-end: adapting the public TrojanStego/Persona-Vectors code to our setting, collecting activations across multiple instruction-pair designs, training and validating a robust probe, running the causal interventions (steering, ablation, interchange), and the axis-relationship analysis. My role is to unblock them when they're stuck, including looking directly at their code, reviewing progress in weekly 1:1s, helping them understand and interpret results as they come in, and helping decide which direction to pursue next (e.g. which causal test to run, how to read an ambiguous result). I'll also help draft the final write-up/paper, including how to position it against very recent related work.

1. Proficient in Python and PyTorch; has trained or fine-tuned a transformer model before
2. Comfortable with, or has used, model internals tooling (TransformerLens, nnsight, or raw HuggingFace hooks), or has completed an interpretability course/program (e.g., ARENA) and can ramp up quickly.
3. Basic linear algebra intuition (dot products, projections, cosine similarity) is sufficient to understand linear probing.
4. Comfortable reading and extending someone else's codebase; this project builds on public code from recent papers rather than starting from scratch.
5. Evidence of independent research or debugging ability: a past project, internship, open-source contribution, or course capstone involving real code, not just coursework exercises.