The course introduces the basic concepts and algorithms related to sequential decision making. Based on the type of feedback received by a system, two types of sequential learning paradigms will be introduced: online learning (full information) and multi-armed bandits (partial information). Algorithms and applications relevant to each paradigm will be discussed.
12 weeks of coursework, weekly online assignments, 2 in-person invigilated quizzes, 1 in-person invigilated end term exam. For details of standard course structure and assessments, visit Academics page.
For details of standard course structure and assessments, visit
Academics
page.
Module 1
Introduction to Online Learning, Halving algorithm, Online Machine Learning; Perceptron and Winnow, Intro to Regret; Online learning with expert advice -Hedge algorithm Online linear optimization, Online convex optimization; Online learning summary
Module 2
Introduction to Multi armed Bandits Adversarial Bandits - EXP3 algorithm, Contextual MAB - EXP4 algorithm, Stochastic MAB, Epsilon Greedy, Explore then commit, Stochastic MAB, UCB, Thompson Sampling, Stochastic MAB - Linear Bandits LinUCB algorithm; MAB summary
The following are the suggested books for the course:
Bubeck S. Introduction to online optimization. Lecture notes. 2011 Dec 14;2:1-86.
Bubeck, S. and Cesa-Bianchi, N., 2012. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning, 5(1), pp.1-122.
About the Instructors
Arun Rajkumar
Assistant Professor,
Department of Data Science and AI,
IIT Madras
I am currently an Assistant Professor at the Data Science and AI department of IIT Madras. Prior to joining IIT Madras, I was a research scientist at the Xerox Research Center (now Conduent Labs), Bangalore for three years. I earned my Ph.D from the Indian Institute of Science where I worked on 'Ranking from Pairwise Comparisons'. My research interests are in the areas of Machine learning, statistical learning theory with applications to education and healthcare.
Please use only the above methods for program queries.
Response time: 3 working days. During peak periods, Google
Meet links will be shared. Call wait times may be longer.
