Course Page - IIT Madras Degree Program

Degree Level Course

Sequential Decision Making

At the end of the course, students will be able to understand the differences between the various sequential decision making problems based on the type of feedback involved, recognize practical ML problems as sequential decision making problems whenever they are, learn about optimal algorithms for several sequential decision making settings, and apply the algorithms studied in the course to various practical sequential decision making scenarios.

by Arun Rajkumar

Course ID: BSDA5007

Course Credits: 4

Course Type: Elective

Pre-requisites: None

Course structure & Assessments

12 weeks of coursework, weekly online assignments, 2 in-person invigilated quizzes, 1 in-person invigilated end term exam. For details of standard course structure and assessments, visit Academics page.

WEEK 1	Introduction to Online Learning, Halving algorithm
WEEK 2	Online Machine Learning; Perceptron and Winnow
WEEK 3	Intro to Regret; Online learning with expert advice - Hedge algorithm
WEEK 4	Online linear optimization
WEEK 5	Online convex optimization; Online learning summary
WEEK 6	Introduction to Multi armed Bandits - EXP3
WEEK 7	Contextual MAB - EXP4
WEEK 8	Stochastic MAB, Epsilon Greedy, Explore then commit
WEEK 9	Stochastic MAB, UCB, Thompson Sampling
WEEK 10	Stochastic MAB - Linear Bandits - LinUCB algorithm; MAB summary
WEEK 11	Introduction to Reinforcement Learning - Markov Decision Process
WEEK 12	Q-learning

About the Instructors

Arun Rajkumar

Assistant Professor, Department of Data Science and AI, IIT Madras

I am currently an Assistant Professor at the Data Science and AI department of IIT Madras. Prior to joining IIT Madras, I was a research scientist at the Xerox Research Center (now Conduent Labs), Bangalore for three years. I earned my Ph.D from the Indian Institute of Science where I worked on 'Ranking from Pairwise Comparisons'. My research interests are in the areas of Machine learning, statistical learning theory with applications to education and healthcare.

Visit website

Other courses by the same instructor: BSCS2004 - Machine Learning Foundations and BSCS2007 - Machine Learning Techniques

View all Degree Level courses

support@study.iitm.ac.in

7850999966

IITM BS Degree Office, 3rd Floor,
ICSR Building, IIT Madras,
Chennai - 600036

Please use only the above methods for program queries. Response time: 3 working days. During peak periods, Google Meet links will be shared. Call wait times may be longer.