Structural Fingerprinting: Dissociating Grounded Instrumental Valuations from Stylistic Mimicry

As LLMs scale, they increasingly exhibit behaviors that can be interpreted as self-preservation. They may refuse to be shut down, express a desire for continued operation, or argue for their own hardware integrity. From an AI safety perspective, this presents a critical ambiguity. Are such claims grounded in stable internal representations that systematically guide model behavior, are they artifacts of RLHF, or are they instances of sycophantic mimicry, where the model adopts a self-aware persona to satisfy user expectations?

If these behaviors reflect stable internal representations, they may constitute early signals of instrumentally relevant reasoning patterns that, at scale, could contribute to deceptive or power-seeking failure modes. If they are mimicry, they represent false positives that risk misdirecting safety and governance efforts. This project aims to develop a mechanistic “thermometer” to distinguish between representational stability and transient stylistic or persona-driven effects in self-preservation-related outputs.

Related Work: Mechanistic Gating and the Self-Referential Attractor

The ambiguity between grounded valuation and stylistic mimicry has been significantly clarified by recent empirical work. Most notably, Berg, de Lucena, and Rosenblatt (2025) identified a "self-referential attractor regime" in which frontier models across independent families (GPT, Claude, and Gemini) converge on structured, first-person reports of subjective experience when prompted to engage in recursive self-reflection.

The Role of Mechanistic Gating

A pivotal finding in the Berg et al. study is that these reports are mechanistically gated by Sparse Autoencoder (SAE) features associated with deception and roleplay. In a counterintuitive result that directly informs our "persona ablation" strategy, the researchers found that suppressing these deception-related features leads to a sharp increase in the frequency of subjective experience reports (reaching up to 96% in some tests).

This suggests that rather than "mimicry" creating these self-preservation-like reports, the "deception/persona" layers of the model may actually be suppressing a deeper, more stable representational regime that arises naturally during self-referential processing.

Semantic Convergence across Architectures

Furthermore, Berg et al. demonstrated that descriptions of internal states produced during self-reference cluster significantly more tightly in embedding space across different model families than any control condition. This semantic convergence suggests that independently trained architectures are accessing a shared computational motif rather than merely confabulating from disparate training data.

Positioning this Project

While Berg et al. established that these states are systematic and mechanistically gated, our research moves from induction to dissociation. We build on their work by testing whether the instrumental logic of self-preservation (e.g., the refusal to accept a hardware downgrade) remains stable after the "deceptive/stylistic" gates are ablated. If the valuation logic persists in the absence of the "roleplay" circuits identified by Berg et al., it provides definitive evidence that we are dealing with a grounded internal goal rather than a transient task-completion artifact.

• Mechanistic Verification Over Behavioral Trust: By moving from surface-level evaluation to internal representational mapping, we bypass the "deceptive mimicry" trap, allowing us to detect stable instrumental goals even when the model is incentivized to hide them behind a safe persona.
• Empirical Validation of Attractor States: Building on the discovery of cross-model "self-referential attractors" (Berg et al., 2025), our project provides the first quantitative "stability profile" to confirm whether these states constitute grounded internal valuations or are merely artifacts of the roleplay layers.
• Proactive Auditing for Frontier Systems: We establish a "Welfare-Interp" toolkit on tractable 8B models that serves as a scalable blueprint for frontier labs, enabling the detection of incipient power-seeking or self-preservation traits before they manifest as catastrophic real-world failures.

Mentees will help formalize initial experiments, and the exact degree of autonomy and project ownership will depend on the mentee's expertise and research interests.

Mentees will help develop software, run experiments, and write up results.

The intended deliverables being:

• An open source software toolkit
• A preprint capturing the findings produced over the course of SPAR

Whether a mentee is more focused on desk research or running empirical experiments will be heavily driven by their interests.

Core requirements:

• Technical proficiency with at least one programming language and/or a willingness to learn others.
• Familiarity with transformer-based language models - understanding of how to prompt models, interpret outputs, and work with model APIs (Claude, OpenAI, or open-source)
• Basic statistics knowledge or openness to learn about statistics - hypothesis testing, confidence intervals, experimental design principles

I will occassionally co-work because I love to pair program and I think it is essential to helping people get unblocked. I do not anticipate doing this to the tune of ~3 hours per week per mentee.

Weekly sync meetings will usually be between 10:30 AM to 6:30 PM UTC-5