Deep Learning for Machine Translation

Lecturers: Hassan Sajjad and Fahim Dalvi

In this lecture series, we first cover the basics of statistical machine translation to establish the intuition behind machine translation. We then cover the basics of neural network models - word embedding and neural language model. Finally, we learn an end-to-end translation system based completely on deep neural networks. In the last part of the lecture series, we learn to peek into these neural systems and analyze what they learn about the intricacies of a language like morphology and syntax, without ever explicitly seeing these details in the training data.

Background reading * Python Numpy Tutorial * IPython Tutorial * Linear Aljebra for Machine Learning

Slides

• Lecture 0 - Introduction & Roadmap slides
• Lecture 1 - Language & Translation slides

• Lecture 2 - Language Modeling slides

  • Python tutorial
  • Python tutorial as a PDF [non-editable]

• Lecture 3 - Machine Learning slides

  • Decision Boundary Exercise [iPython/Jupyter Notebook]
  • Optimization functions demonstration [iPython/Jupyter Notebook]

• Lecture 4 - Machine Learning II slides

  • Linear Classifier with MSE [iPython/Jupyter Notebook]
  • Linear Classifier with Softmax and Cross Entropy [iPython/Jupyter Notebook]

• Lecture 5 - Machine Learning and Neural Networks slides

  • Efficient Linear Classifier with Softmax and Cross Entropy [iPython/Jupyter Notebook]
  • Neural Network [iPython/Jupyter Notebook]
  • Neural Network with Keras toolkit [iPython/Jupyter Notebook]

• Lecture 6 - Neural Network Language Models slides

  • Language Modeling with Keras [iPython/Jupyter Notebook]

• Lecture 7 - Sequence to Sequence slides

• Lecture 8 - Practical Neural MT slides

• Lecture 9 - Analysis of Neural MT slides

• Lecture 10 - Recent Advancements slides