Evals Synthesis using Interpretability

While LLM agents are increasingly capable, they still fail in surprising and high-stakes edge cases, particularly in under-tested or atypical scenarios. Exhaustive evaluation across all possible inputs is intractable. We propose instead to pre-empt these failures by leveraging interpretability to identify likely failure modes hidden within evaluation gaps. Our theory-of-impact is that a model’s internal representational geometry and mechanisms can be used to anticipate failure before deployment. Rather than relying on static benchmarks or fully black-box testing, we analyze the model’s internal structure to predict which untested scenarios are most likely to induce failure, and generate targeted challenge sets accordingly. This creates an automated and scalable evaluation loop that evolves with the models, offering a lightweight but proactive approach to robust testing—a critical yet underexplored priority in technical AI safety, particularly as models grow more opaque and update more frequently. As a scalable case study, we will develop stress tests that systematically vary both language and task. Multilingual settings offer a promising testbed, as LLMs must align intractably many language-specific representations with universal, language-agnostic ones to perform tasks like math, factual recall, and reasoning. Even when models appear fluent, they may underperform on specific tasks, suggesting a disconnect between universal task representations and language-specific surfaces. We hypothesize that this gap can be predicted by analyzing how task representations interact with language-local components. Such misalignments between universal vs. data-specific representations likely underlie broader failure modes beyond multilinguality. We will predict failures from these interactions using several candidate methods, including geometric analysis, rare token prediction, and model invariance. These analyses will inform the design of targeted stress tests and offer broader insight into the limits of model generalization. We will consider our approach to be effective if models perform worse on an adaptively generated challenge set than on existing benchmarks and a random baseline of task-language combinations. We will also investigate if post-training on our challenge sets enables model repair. Our ultimate goal is a self-improving robust evaluation loop: interpretability informs stress testing, which then informs further model development. As models grow more complex, white-box diagnostics will be essential for anticipating and preventing failures.

In the long term, we hope to contribute toward scalable oversight that adapts to rapidly evolving AI systems, and ensures anticipatory safety at frontier scales.

I see two major points of impact:

1. Predicting, rather than monitoring, is active. We can proactively act to prevent failures before they arise in deployment, after impact.
2. Models do not always behave according to human intuition, so current evaluation, which are mostly human driven, may not catch what models find difficult. We can better define safety failure cases like these automatically.

I expect to develop concepts/project directions/proposals with mentees. I expect them to be curious and self-motivated, who are excited by their own ideas. We can then craft initial, broader sets of investigations together based on their interests, fail fast, and come up with a solidifying thesis that has a high potential to succeed.

With guidance, mentees will ideate with me. Experiments will be mostly performed by the mentees, and analyses/outcomes can be reviewed again in detailed discussion with me and others that have relevant expertise in the particular technical direction we settle on.

I'd like to especially encourage researchers from underrepresented/nontraditional backgrounds.

• proficient with Python and standard ML programming (pytorch, huggingface, etc.)
• have done interpretability work with model internals: attention, circuits, activation steering etc. (at least 50 hours, preferrably 100 hours)
• proficient knowledge of math would be a plus! but not necessary.

N/A