Develop a novel interactive coding agent based on a planner distillation codebase.
About the project
This project will leverage a finetuning codebase in order to implement a coding agent that combines theory of mind about user intent with constraints about coding specifications and best practices. The goal of the project is to study the effectiveness and robustness of a novel agent design that relies on distilling verified behavior traces from a hierarchical planner into a model. The approach leverages a neurosymbolic planner that iteratively refines high-level plans until the resulting behavior satisfies a compositional specification. It implements a planner-distillation approach in order to effectively imitate this behavior with a hierarchical neural network policy.
The approach will integrate the goal tracking system from "Open-Universe Assistance Games" (https://arxiv.org/abs/2508.15119) with a forthcoming extension of "Guided Imitation of Task and Motion Planning" (https://arxiv.org/abs/2112.03386) to belief space. I'm flexible about which domain we apply it to, but I suspect that the Stateful-SWE domain from "TOM-SWE: User Mental Modeling For Software Engineering Agents" (https://arxiv.org/abs/2510.21903) would be a valuable and interesting application.
Theory of change
This project considers two critical problems for AI safety: 1) developing agents that reliably implement specifications and adhere to safety constraints on their behavior; and 2) agents that specifically follow constraints that depend on their beliefs about the world. This approach explores both of these. I believe it provides an interesting and novel alternative to RL-based training paradigms for alignment, which are inherently brittle and can respond unpredictably to increased compute resources.
Your role
This is largely described above in the project details. Specifically, I expect that mentees will spend their time across the following: developing new agent specifications, iterating on data generation and agent distillation, agent testing and iteration, and agent evaluation.
Prerequisites
- Strong engineering skills and experience with Python
- Experience with finetuning and model training for moderately sized models
- Familiarity with symbolic planning methods (e.g., STRIPS, PDDL) or a capacity and desire to learn quickly
Location preference
I will mostly be able to meet 9-5 ET, but am flexible on timezone outside of that
Application question(s)
Please plan out an initial two-week work schedule for the proposed project. What are the key goals you'd hope to accomplish? What challenges do you anticipate? How do you expect that you would spend most of your time? What would you need help/guidance on?
About the mentor
Dylan is an Associate Professor of EECS at MIT and leads the Algorithmic Alignment Group in the Computer Science and Artificial Intelligence Laboratory (CSAIL). His research develops methods to ensure that AI systems behave in ways that align with the goals and values of their human users and society as a whole, a concept known as “AI alignment.” His group works on alignment challenges in multi-agent systems, human-AI teams, and societal oversight of machine learning. Their goal is to enable the safe, beneficial, and trustworthy deployment of AI in real-world settings.