In my previous SPAR AI project, our team built agents that automatically explain internal features in language models using tools like Sparse Autoencoders (SAEs) and Cross Layer Transcoders (CLTs). The next logical step is to scale this from single features to entire circuits.
In this follow up project, we will build AI agents that read attribution graphs, identify important subcircuits, and describe what they are doing. The goal is to turn dense graphs of thousands of nodes into human friendly explanations and targeted experiments.
About the project
Over the last year, frontier models have become much better at “agency”. They can plan, use tools, write and run code, search the web, and iterate on their own ideas. That looks a lot like what interpretability researchers do when we analyze circuits in a model: read graphs, form hypotheses, write code to test them, run ablations, then refine our explanation.
In my previous SPAR project, we asked: can we turn this workflow into an AI agent for single features. The team built agents that:
- inspect activations and statistics of SAE or CLT features
- propose candidate explanations
- run small experiments to test and refine those explanations
This worked well for feature level analysis. However, many of the most interesting behaviors in language models are implemented by circuits that span many features and layers. Recent work on Cross Layer Transcoders and circuit tracing lets us build attribution graphs that show how information flows through a model for a single prompt. These graphs are rich but also large and hard to navigate.
This project focuses on that next step: AI agents for circuit interpretability.
We will build agents that:
- explore attribution graphs and find the most important paths and subcircuits
- summarize what specific subgraphs seem to be computing
- design and run causal tests that validate or falsify those explanations
- produce structured explanations that a human researcher can quickly review
You can get an intuition for these graphs at Neuronpedia’s Circuit Tracer demo. Your job will be to help turn that interactive visualization into something an AI agent can understand, reason about, and experiment on.
References:
https://alignment.anthropic.com/2025/automated-auditing/ Cunningham, H., Ewart, A., Riggs, L., Huben, R., & Sharkey, L. (2023). Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600 Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., ... & Olah, C. (2023). Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2. Ameisen, E., Lindsey, J., Pearce, A., Gurnee, W., Turner, N. L., Chen, B., ... & Batson, J. (2025). Circuit tracing: Revealing computational graphs in language models. Transformer Circuits Thread, 6. Lindsey, J. et al. Sparse Crosscoders for Cross-Layer Features and Model Diffing, Transformer Circuits Thread Bills, S. et al. Language models can explain neurons in language models, OpenAI Juang, C. et al Open Source Automated Interpretability for Sparse Autoencoder Features. Eleuther AI https://www.neuronpedia.org/gemma-2-2b/graph
Theory of change
Mechanistic Interpretability hopes to improve AI safety by auditing model internals rather than relying on black box analysis. We now have methods to disentangle representations into interpretable features, and we can use these features to trace entire circuits and explain why the model predicted a certain word.
However, frontier AI models have billions of circuits and the current, manual approach is not scalable. This project aims at automating the manual parts of the interpretability pipeline using AI agents. If successful, safety-testing frontier AI models using the circuit discovery pipeline would become more tractable.
Your role
Mentees will own the hands-on work of the project, and I will provide strategic guidance and unblock any challenges that arise.
Because the project is ambitious, I encourage everyone to pair up or form small teams; collaboration makes the work faster, richer, and more enjoyable. Mentees remain free to steer their own projects and can choose how much guidance they want, and I will tailor my support so each person gets the most from the program given their skills, commitment, and goals.
Prerequisites
- highly proficient using Python
- have implemented a transformer from scratch
One or more of the following:
- worked with pytorch hooks, transformerlens or nnsight to get model activations
- web development skills / happy to vibe code basic web apps
- experience with prompt engineering
- used an LLM API (OpenAI, Anthropic, etc)
Application question(s)
Understanding a circuit with Neuronpedia Circuit Tracer
In this task you will use Neuronpedia’s Circuit Tracer to explore one concrete circuit in a language model. The goal is not to be correct at all costs, but to show how you think: how you form hypotheses, read a graph, and test your ideas.
You can start from the Gemma 2 2B Circuit Tracer here: https://www.neuronpedia.org/gemma-2-2b/graph
Feel free to click around, try different prompts, and pick one that you actually find interesting. Please do not spend more than about 30 minutes in total.
Question 1 Which prompt did you choose, and why?
- Paste the Neuronpedia URL for the exact Circuit Tracer view you analyzed
- Quote the prompt and briefly describe what the model is being asked to do
- Explain why you think this prompt is interesting to analyze mechanistically
- Write down your initial hypothesis about what kind of circuit you expect to see
(max 150 words)
Question 2 Read the circuit.
- Spend some minutes exploring the graph for your prompt.
- Group features into supernodes to create a high-level circuit.
- What are the main information flows you see from input to output?
- Can you describe a high level circuit that the model might be using for this prompt?
Try to give a story like “First the model does X here, then this subcircuit seems to compute Y, and finally this path combines them into Z.”
(max 250 words)
Question 3 Test your hypothesis with a causal experiment.
- Use Circuit Tracer’s steering tools on at least one supernode that you believe is important for the behavior you described.
- What supernode did you pick and why?
- What steering intervention did you apply? (for example: increase or decrease its activation, or change its behavior in some direction)
- What changed in the model’s output?
Briefly describe what you observed, including at least one surprising or confusing aspect if there was any.
(max 200 words)
About the mentor
Georg Lange is an independent researcher working on Mechanistic Interpretability for LLMs. His projects aim to turn a loose collection of LLM interpretability techniques into a unified pipeline that runs from feature discovery to algorithm-level explanations of model behavior. His interests span dictionary learning, sparse autoencoders, automated interpretability, circuit discovery, and the study of reasoning models.
Previously, he was a MATS scholar and worked with Alex Makelov and Neel Nanda on Sparse Autoencoders and Distributed Alignment Search for feature detection and activation patching. He studied Artificial Intelligence at the University of Amsterdam, where he worked on brain-like interpretable spatiotemporal Computer Vision models. Further, he was a graduate student of Cognitive Neuroscience at CUNY and worked on Reinforcement Learning, Decision Making, and Reward Sensitization in mice.