Goals of this course

• You will gain insight into the capabilities and limitations of large-scale models, including their use in multimodal settings.
• You will learn how to apply state-of-the-art machine learning models to solve real-world problems.
• You will develop practical skills in building LLM-driven chat interfaces.
• You will learn techniques to improve generative model outputs using retrieval-augmented generation.

Topics

• Data Modalities and Storage
• Representation Learning
• Text Models
• Discriminative Image Models
• Joint Embedding Methods
• Generative Image Models
• Retrieval Systems
• Evaluation Methods

Recommended and mandatory Prerequisites

• Data mining: basic concepts
• Introduction to machine learning
• Proficiency in Python programming, optionally with some experience in PyTorch.
• A modern laptop is required, such as the Lenovo T14 Gen 2 (2022).