I am a CS PhD candidate at the Center for AI Safety at Stanford University, under the guidance of Prof. Clark Barrett. I want to understand how values form. In reinforcement learning, decision theory, and game theory, the utility function is fixed at the outset and sits outside the agent, whereas the way values form in organisms, within a lifetime and across lifetimes, is highly structured. My view of AI alignment follows from this: values should be raised, not specified. Most recently I gave a definition of exploration that is not relative to any terminal goal: one policy explores at least as much as another if running it eventually teaches you what would have happened under the other. I used it to show where standard intrinsic rewards fall short and when they succeed. I am now asking whether placing care into an AI’s environment leads to robustly good values.
Before this, I trained language models to use their chain-of-thought as a natural-language hidden state, so that the reasoning is load-bearing for prediction (ICLR 2026). I came to alignment from formal verification and programming languages at Stanford, after studying interactive theorem proving at CMU with Simon DeDeo, where we published in Cognition on why abduction is needed to believe the corpus of known mathematics. I majored in computer science and electrical engineering at MIT.
My primary character trait is curiosity, and I really love math.
Ph.D. in Computer Science (emphasis in Artificial Intelligence), present
Stanford University
B.S. in Computer Science and Electrical Engineering, 2018
Massachusetts Institute of Technology
Course Organizer
Course Organizer
Discussion Facilitator
Discussion Facilitator
Intrinsic rewards are designed to guide exploration in reinforcement learning by assigning value to an agent’s experience, for example through prediction error or learning progress. However, maximizing these rewards need not produce the most informative experience available. We propose a formal criterion for exploration based on the comparison of statistical experiments. It compares policies (rules for choosing actions from past actions and observations) by the counterfactual information they acquire: how well their histories can substitute for experience under alternative policies. We construct a single, simple environment in which specified count-based, prediction-error, empowerment, and information-gain objectives have maximizing policies that are strictly dominated under this criterion. We explain these failures and establish conditions under which existing intrinsic rewards successfully encourage optimal exploration. We also construct an objective that assigns a higher value whenever exploration strictly improves under our criterion.
For STS 10SI – Intro to AI Alignment in Winter 2023 I talked eliciting latent knowledge, cooperative inverse reinforcement learning, and shard theory. First I introduce the ELK problem as formulated by Paul Christiano and I go through the main proposals and counterexamples from the ELK document. Then we introduce inverse reinforcement learning, cooperative inverse reinforcement learning, and their relationship to the alignment problem. We proceed with a discussion of whether humans can be thought of as having utility functions, leading into a conversation about Shard theory. Lastly, we talk about similarities between the limbic-cortex relationship and the alignment problem, and frame utility functions as a story the brain tells about itself. Slides here.
A presentation on ontology maps as a framework for AI alignment, exploring how mappings between agent representations can be used to ensure AI systems retain human-compatible concepts and values.
How can we create intelligent systems that retain and expand into that which we find valuable in the universe? During this talk I will present my own thoughts based on ontology mapping, and I will communicate why I believe that mathematicians who think about systemic interactions are in an especially good place to answer this important question. I will start with a framework in which read-eval-print loops (repls) form a basis for reasoning about agents and computation in general. Then I will build ontology maps as an alignment framework on top of repls, and discuss its implications. Lastly I will invite discussion on what we might want out of the future, and talk about my thoughts on the central role of communication and its relation to both repls and ontology maps.