4.75 (6 reviews)
☑ Deep Learning for Natural Language Processing
☑ Multi-Layered Perceptrons (MLPs)
☑ Word embeddings
☑ Recurrent Models: RNNs, LSTMs, GRUs and variants
☑ DL for NLP
This course is a part of "Deep Learning for NLP" Series. In this course, I will introduce basic deep learning concepts like multi-layered perceptrons, word embeddings and recurrent neural networks. These concepts form the base for good understanding of advanced deep learning models for Natural Language Processing.
The course consists of three sections.
In the first section, I will talk about Basic concepts in artificial neural networks like activation functions (like ramp, step, sigmoid, tanh, relu, leaky relu), integration functions, perceptron and back-propagation algorithms. I also talk about what is deep learning, how is it related to machine learning and artificial intelligence? Finally, I will talk about how to handle overfittting in neural network training using methods like regularization, early stopping and dropouts.
In the second section, I will talk about various kinds of word embedding methods. I will start with basic methods like Onehot encoding and Singular Value Decomposition (SVD). Next I will talk about the popular word2vec model including both the CBOW and Skipgram methods. Further, I will talk about multiple methods to make the softmax computation efficient. This will be followed by discussion on GloVe. As special word embedding topics I will cover Cross-lingual embeddings. Finally, I will also talk about sub-word embeddings like BPE (Byte Pair Encoding), wordPiece, SentencePiece which are popularly used for Transformer based models.
In the third session, I will start with general discussion on ngram models. Next I will briefly introduce the neural network language model (NNLM). Then we will spend quite some time understanding how RNNs work. We will also talk about RNN variants like BiRNNs, Deep BiRNNs. Then I will discuss the vanishing and exploding gradients problem. This will be followed by details of the LSTMs and GRUs architectures.
Multi-Layered Perceptrons (MLPs)
Why do we need Artificial Neural Networks (ANNs)?
Artificial neuron: activation/integration function, softmax, perceptron
Why do we need Multi-Layered Perceptrons?
What is deep learning?
How does back-propagation work?
Overfitting, dropout and regularization
Onehot encoding and SVD
word2vec (CBOW, Skipgram)
Efficient Softmax approximations
Sampling-based approximations for softmax
Cross-lingual word embedding models
Sub-word level embeddings
Recurrent Models: RNNs, GRUs, LSTMs, variants.
Traditional n-gram language models and NNLM
Recurrent Neural Networks: RNNs
RNNs for Image captioning
Bidirectional RNNs, Stacked RNNs, Vanishing gradients problem
Long Short-Term Memory Networks: LSTMs
Gated Recurrent Units: GRUs
Thank you Dr Manish - Great introduction on RNNs, LSTMS & GRU Bidirectional concepts. You have made very simple to understand these concepts and intuition behind.