Formulation and solution of sequential decision problems under uncertainty as a foundation for artificial intelligence, operations research, and economics. Finite-horizon, discounted, and average reward objectives. Optimization via value iteration, policy iteration, linear programming, and reinforcement learning algorithms. Semi-Markov decision processes. Multi-armed bandits and the Gittin's index theorem. Use of partial state information. Homework assignments involve a combination of analytic and computational exercises.
3 units · Letter or Credit/No Credit
Formulation and solution of sequential decision problems under uncertainty as a foundation for artificial intelligence, operations research, and economics. Finite-horizon, discounted, and average reward objectives. Optimization via value iteration, policy iteration, linear programming, and reinforcement learning algorithms. Semi-Markov decision processes. Multi-armed bandits and the Gittin's index theorem. Use of partial state information. Homework assignments involve a combination of analytic and computational exercises.
Offered in Autumn 2026 at Stanford University.