Master Seminar on Word Embedding Spaces

Material for the seminar "Word Embedding Spaces", Heinrich-Heine-University, Düsseldorf, Germany. This seminar took place at HHU:

• Summer term 2022
• Summer term 2023 (in preparation)

Outline

• presentations for slides
• notes for the seminar program, additional write-up and lists of references,
• exercises for practical tutorials (work in progress)

Syllabus

• Static Word Embeddings
  • Frequency-based Methods
  • word2vec, GloVe, fastText
• Contextual Word Embeddings
  • RNNs, LSTMs (e.g., ElMo)
  • Attention and Transformers
  • Masked Language Models (e.g., BERT, RoBERTa)
  • Autoregressive Language Models (e.g., GPT, GPT-2, GPT-3)
  • Contrastive Learning, Sentence Embeddings
• Additional topics
  • Evaluation, Geometry of the Embedding Space
  • Bias, Sentiment
  • Multilingual Embeddings
  • Multimodal Embeddings
    • CLIP: a common embedding space for words and images
• Dialogue Systems
  • Task-oriented Dialogue System Components
  • Aligned Language Models: InstructGPT, ChatGPT, Sparrow
• Representation Learning, Geometric Deep Learning, Topological Data Analysis
  • Hyperbolic Embeddings
  • Singularities and Topological Polysemy
  • Riemannian Manifold Learning
  • (Topologically motivated) Dimensionality Reduction

Schedule

Summer term 2023

Seminar dates: 05.04.2023 12.04.2023 19.04.2023 26.04.2023 03.05.2023 10.05.2023 17.05.2023 24.05.2023 31.05.2023 07.06.2023 14.06.2023 21.06.2023 28.06.2023 05.07.2023 12.07.2023

Summer term 2023 Schedule on Google Sheets

Summer term 2022

Summer term 2022 Schedule on Google Sheets