Modern NLP using Deep Learning

You will learn the newest state-of-the-art Natural language processing (NLP) Deep-learning approaches.

You will

1. Get state-of-the-art knowledge regarding

   1. NMT

   2. Text summarization

   3. QA

   4. Chatbot

2. Validate your knowledge by answering short and very easy 3-question queezes of each lecture

3. Be able to complete the course by ~2 hours.

Syllabus

1. Neural machine translation (NMT)

   1. Seq2seq
      A family of machine learning approaches used for natural language processing.

   2. Attention
      A technique that mimics cognitive attention.

   3. NMT
      An approach to machine translation that uses an artificial neural network to predict the likelihood of a sequence of words, typically modelling entire sentences in a single integrated model.

   4. Teacher-forcing
      An algorithm for training the weights of recurrent neural networks (RNNs).

   5. BLEU
      An algorithm for evaluating the quality of text which has been machine-translated from one natural language to another.

   6. Beam search
      A heuristic search algorithm that explores a graph by expanding the most promising node in a limited set.

2. Text summarization

   1. Transformer
      A deep learning model that adopts the mechanism of self-attention, differentially weighting the significance of each part of the input data.

3. Question Answering

   1. GPT-3
      An autoregressive language model that uses deep learning to produce human-like text.

   2. BERT
      A transformer-based machine learning technique for natural language processing (NLP) pre-training developed by Google.

4. Chatbot

   1. LSH
      An algorithmic technique that hashes similar input items into the same "buckets" with high probability.

   2. RevNet
      A variant of ResNets where each layer's activations can be reconstructed exactly from the next layer's.

   3. Reformer
      Introduces two techniques to improve the efficiency of Transformers.

Resources

• Wikipedia

• Coursera