Udemy

Platform

English

Language

Other

Category

Deep Learning for NLP - Part 1

Part 1: Multi-Layered Perceptrons, Word Embeddings and Recurrent neural networks

4.75 (6 reviews)

Deep Learning for NLP - Part 1

Students

3.5 hours

Content

Jul 2021

Last Update
Regular Price

SKILLSHARE
SkillShare
Unlimited access to 30 000 Premium SkillShare courses
30-DAY FREE TRIAL

What you will learn

Deep Learning for Natural Language Processing

Multi-Layered Perceptrons (MLPs)

Word embeddings

Recurrent Models: RNNs, LSTMs, GRUs and variants

DL for NLP


Description

This course is a part of "Deep Learning for NLP" Series. In this course, I will introduce basic deep learning concepts like multi-layered perceptrons, word embeddings and recurrent neural networks. These concepts form the base for good understanding of advanced deep learning models for Natural Language Processing.

The course consists of three sections.

In the first section, I will talk about Basic concepts in artificial neural networks like activation functions (like ramp, step, sigmoid, tanh, relu, leaky relu), integration functions, perceptron and back-propagation algorithms. I also talk about what is deep learning, how is it related to machine learning and artificial intelligence? Finally, I will talk about how to handle overfittting in neural network training using methods like regularization, early stopping and dropouts.

In the second section, I will talk about various kinds of word embedding methods. I will start with basic methods like Onehot encoding and Singular Value Decomposition (SVD). Next I will talk about the popular word2vec model including both the CBOW and Skipgram methods. Further, I will talk about multiple methods to make the softmax computation efficient. This will be followed by discussion on GloVe. As special word embedding topics I will cover Cross-lingual embeddings. Finally, I will also talk about sub-word embeddings like BPE (Byte Pair Encoding), wordPiece, SentencePiece which are popularly used for Transformer based models.

In the third session, I will start with general discussion on ngram models. Next I will briefly introduce the neural network language model (NNLM). Then we will spend quite some time understanding how RNNs work. We will also talk about RNN variants like BiRNNs, Deep BiRNNs. Then I will discuss the vanishing and exploding gradients problem. This will be followed by details of the LSTMs and GRUs architectures.


Content

Multi-Layered Perceptrons (MLPs)

Introduction

Why do we need Artificial Neural Networks (ANNs)?

Artificial neuron: activation/integration function, softmax, perceptron

Why do we need Multi-Layered Perceptrons?

What is deep learning?

How does back-propagation work?

Overfitting, dropout and regularization

Summary

Word Embeddings

Introduction

Onehot encoding and SVD

word2vec (CBOW, Skipgram)

Efficient Softmax approximations

Sampling-based approximations for softmax

GloVe

Cross-lingual word embedding models

Sub-word level embeddings

Summary

Recurrent Models: RNNs, GRUs, LSTMs, variants.

Introduction

Traditional n-gram language models and NNLM

Recurrent Neural Networks: RNNs

RNNs for Image captioning

Bidirectional RNNs, Stacked RNNs, Vanishing gradients problem

Long Short-Term Memory Networks: LSTMs

Gated Recurrent Units: GRUs

Summary


Reviews

S
Suryanarayana22 May 2021

Thank you Dr Manish - Great introduction on RNNs, LSTMS & GRU Bidirectional concepts. You have made very simple to understand these concepts and intuition behind.


4005750

Udemy ID

4/25/2021

Course created date

5/23/2021

Course Indexed date
Bot
Course Submitted by