Human preference data has become crucial to the success of Machine Learning (ML) systems in many application domains, from personalization to post-training of language models. As ML systems are more and more widely deployed, understanding models, methods, and algorithms for learning from preference data becomes important for both scientists and practitioners. This course covers learning from preferences in supervised, active, and reinforcement/assistance settings, and covers aspects specific to preference data, such as preference heterogeneity and aggregation, interpretation of human feedback, and privacy. In coding tasks, students implement supervised reward modeling and assistance games.
3 units · Letter or Credit/No Credit
Human preference data has become crucial to the success of Machine Learning (ML) systems in many application domains, from personalization to post-training of language models. As ML systems are more and more widely deployed, understanding models, methods, and algorithms for learning from preference data becomes important for both scientists and practitioners. This course covers learning from preferences in supervised, active, and reinforcement/assistance settings, and covers aspects specific to preference data, such as preference heterogeneity and aggregation, interpretation of human feedback, and privacy. In coding tasks, students implement supervised reward modeling and assistance games.
Offered in Autumn 2026 at Stanford University.