Degree Level Course
Large Language Models
Understanding the Transformer architecture Understanding the concept of pretraining and fine-tuning language models Compare and contrast different types of tokenizers like BPE, wordpiece, sentencepiece Understanding different LLMs architectures: encoder-decoder, encoder-only, decoder-only Exploring common datasets like C4,mc4,Pile, Stack and so on Addressing the challenges of applying vanilla attention mechanisms for long range context windows. Apply different types of fine-tuning techniques to fine-tune large language models

Course ID: BSCS5001
Course Credits: 4
Course Type: Elective
Pre-requisites: BSCS3004 - Deep Learning