Explore what data could differentially train future models to conduct useful AI safety research.
About the project
Pilot a low-overhead system for collecting data from AI safety organizations that could help train future models to conduct better safety research.
Theory of change
As models become better at automated research, the data available for post-training may shape which kinds of research they can perform well. AI safety organizations may already generate valuable data through research meetings, feedback, project development, and internal discussions, but collecting it raises privacy concerns and can create substantial overhead.
Mentees will develop and begin piloting a smooth system for onboarding an organization into data collection. This may involve designing consent and privacy processes, identifying useful data, building tools such as Claude plugins, and working with projects such as Alignment-Hive.
Your role
Mentees will design an onboarding and consent process, build or configure collection tools, and launch or prepare a pilot. They may do independent research or consult researchers at AI safety organizations on questions about which data would be most useful.
Prerequisites
Familiarity with machine learning, model post-training, or AI safety research would be useful. Strong conceptual research and writing skills are more important than software engineering experience.
Application question(s)
What type of data might help models conduct better AI safety research?
Why might this data differentially improve safety research rather than research capabilities in general?
About the mentor
Alec Harris
Pivotal Research Extension
Alec Harris is a researcher with the Pivotal Research Extension.