(PhD) Dynamic Programming

This course offers an advanced introduction Markov Decision Processes (MDPs)-a formalization of the problem of optimal sequential decision making under uncertainty-and Reinforcement Learning (RL)-a paradigm for learning from data to make near optimal sequential decisions. The first part of the course will cover foundational material on MDPs. We'll then look at the problem of estimating long run value from data, including popular RL algorithms like temporal difference learning and Q-learning. The final part of the course looks at the design and analysis of efficient exploration algorithms, i.e. those that intelligently probe the environment to collect data that improves decision quality. This a doctoral level course. Students should have experience with mathematical proofs, coding for numerical computation, and the basics of statistics, optimization, and stochastic processes.