NLP Demystified 14: Machine Translation With Sequence-to-Sequence and Attention

Course playlist: https://www.youtube.com/playlist?list=PLw3N0OFSAYSEC_XokEcX8uzJmEZSoNGuS Whether it's translation, summarization, or even answering questions, a lot of NLP tasks come down to transforming one type of sequence into another. In this module, we'll learn to do that using encoders and decoders. We'll then look at the weaknesses of the standard approach, and enhance our model with Attention. In the demo, we'll build a model to translate languages for us. Colab notebook: https://colab.research.google.com/github/futuremojo/nlp-demystified/blob/main/notebooks/nlpdemystified_seq2seq_and_attention.ipynb Timestamps 00:00:00 Seq2Seq and Attention 00:00:37 Seq2Seq as a general problem-solving approach 00:02:17 Translating language with a seq2seq model 00:05:53 Machine translation challenges 00:09:07 Effective decoding with Beam Search 00:13:04 Evaluating translation models with BLEU 00:16:23 The information bottleneck 00:17:56 Overcoming the bottleneck with Attention 00:22:39 Additive vs Multiplicative Attention 00:26:47 [DEMO] Neural Machine Translation WITHOUT Attention 00:50:59 [DEMO] Neural Machine Translation WITH Attention 01:04:53 Attention as information retrieval This video is part of Natural Language Processing Demystified --a free, accessible course on NLP. Visit https://www.nlpdemystified.org/ to learn more.