ECS 282: Advanced Natural Language Processing

Expanded Course Description

Introducing foundational technologies for building NLP systems based on neural language models. Discussing cutting-edge research advancements on large (multi-modal) language models. The project will be exploratory, involving one proposal and one final paper. The following is a list of topics covered in this course.

1. Word representation: discrete and distributional representation for words.

2. Language models: statistical and basic neural language models.

3. Sentence representation: discrete, neural sentence representation and sentence pair representation.

4. Text classification and retrieval: neural text classification, textual entailment, text retrieval and their learning objectives.

5. Subword and contextualized representation: neural representation for handling out-of-vocabulary tokens, contextually varied representation.

6. Transformers: basics of Transformers, pre-trained Transformer language models.

7. Question answering and summarization: basic techniques for open-domain question answering, extractive and abstractive summarization.

8. Large language models (LLMs): LLM development paradigms, inference paradigms, adaptation and emergent challenges.