Applications Open now for September 2024 Batch | Applications Close: Sep 15, 2024 | Exam: Oct 27, 2024

Applications Open now for September 2024 Batch | Applications Close: Sep 15, 2024 | Exam: Oct 27, 2024

Degree Level Course

Speech Technology

by Prof. S. Umesh , Hema A Murthy

Course ID: BSEE4001

Course Credits: 4

Course Type: Elective

Pre-requisites: None

What you’ll learnVIEW COURSE VIDEOS

To understand the concepts of speech and speech technologies, and to apply them to real-world scenarios
To gain hands-on experience of the relevant toolkits used for speech processing

Course structure & Assessments

12 weeks of coursework, weekly online assignments, 2 in-person invigilated quizzes, 1 in-person invigilated end term exam. For details of standard course structure and assessments, visit Academics page.

WEEK 1 Review of Signals and Systems, Continuous time signals and transforms Discrete time signals, Discrete Fourier transform, Autocorrelation and Cross-Correlation
WEEK 2 Acoustic Feature Analysis of Speech Signals I, II Gaussian mixture models (GMM), universal background model (UBM-GMM), singular value decomposition (SVD)
WEEK 3 Hidden Markov model (HMM), Examples of HMM based approach for ASR, TTS, speaker diarization Information bottleneck (IB) based clustering for diarization
WEEK 4 Introduction and History of ASR and TTS Components of ASR: Acoustic Modelling, Punctuation Model (Lexicon) and language modelling (N-Gram Language models)
WEEK 5 HMMs for Acoustic Modelling - Monophone, Triphone Speech Synthesis: unit selection, statistical parametric synthesis (HTS)
WEEK 6 Neural networks for building speech technologies NN for Acoustic Modelling - Hybrid modelling- Hybrid-NN: DNN,CNN,TDNN
WEEK 7 End-to-End Approaches I: CTC, Encoder-decoder Architecture E2E with RNN
WEEK 8 Applications to ASR and TTS End-to-End Approaches II
WEEK 9 Encoder-decoder Architecture E2E with transformers for ASR and TTS Interesting Problems
WEEK 10 Speaker recognition/verification: with ivector, xvector Speaker diarization: using x-vector
WEEK 11 Speaker adaptation: (revisit i, x vectors) and introduce s-vectors. Code Switched Speech recognition; Speech Translation
WEEK 12 Singing voice synthesis; voice conversion; generic voice synthesis
+ Show all weeks

Prescribed Books

The following are the suggested books for the course:

L R Rabiner and R W Schafer, "Theory and Application of Digital Speech Processing", PH, Pearson, 2011.

L R Rabiner, B-H Juang and B Yegnanarayana, "Fundamentals of Speech Recognition", Pearson, 2009 (Indian subcontinent adaptation).

Xuedong Huang, Alex Acero, Hsiao-wuen Hon, "Spoken Language Processing: A guide to Theory, Algorithm, and System Development", Prentice Hall PTR, 2001.


Thomas Quatieri, "Discrete-time Speech Processing: Principles and Practice", PH, 2001.

Rabiner and Schafer, "Digital Processing of Speech Signals", Pearson Education, 1993.

Recent research papers

About the Instructors

Prof. S. Umesh
Professor, Department of Electrical Engineering, Indian Institute of Technology, IIT Madras

S. Umesh is a  Professor of Electrical Engineering at IIT-Madras. He completed his PhD from the University of Rhode Island,USA and his PostDoctoral Fellowship from the City University of New York. He has also been a visiting researcher at AT&T Research Laboratories, USA; at Machine Intelligence Laboratory Cambridge University Engineering Department, UK and the Department of Computer Science, RWTH-Aachen, Germany.

...  more

He is a recipient of the AICTE Career Award for Young Teachers in 1997 and the Alexander von Humboldt Research Fellowship in 2004.  During his stint at Cambridge University in 2004, he was part of the U.S. DARPA's Effective, Affordable Reusable Speech-to-text (EARS) programme. Similarly in 2005 he was part of the RWTH-Aachen's TC-STAR project for transcription of speech from European Parliament's Plenary Sessions. Between 2010-2016, he led a multi-institution consortium to develop ASR systems in Indian languages in the agriculture domain which was funded by MeiTY. He is currently leading the ASR efforts for the Natural Language Translation Mission managed by the Office of Principal Scientific Adviser of Govt. of India.


Hema A Murthy
Professor, Department of Computer Science and Engineering, IIT Madras

Faculty at the Department of Computer Science and Engineering, Indian Institute of Technology Madras.