Advanced AI: Deep Reinforcement Learning in Python

The Complete Guide to Mastering Artificial Intelligence using Deep Learning and Neural Networks

4.63 (5602 reviews)
Udemy
platform
English
language
Data Science
category
Advanced AI: Deep Reinforcement Learning in Python
39,756
students
10.5 hours
content
Apr 2024
last update
$24.99
regular price

What you will learn

Build various deep learning agents (including DQN and A3C)

Apply a variety of advanced reinforcement learning algorithms to any problem

Q-Learning with Deep Neural Networks

Policy Gradient Methods with Neural Networks

Reinforcement Learning with RBF Networks

Use Convolutional Neural Networks with Deep Q-Learning

Understand important foundations for OpenAI ChatGPT, GPT-4

Why take this course?

Ever wondered how AI technologies like OpenAI ChatGPT and GPT-4 really work? In this course, you will learn the foundations of these groundbreaking applications.

This course is all about the application of deep learning and neural networks to reinforcement learning.

If you’ve taken my first reinforcement learning class, then you know that reinforcement learning is on the bleeding edge of what we can do with AI.

Specifically, the combination of deep learning with reinforcement learning has led to AlphaGo beating a world champion in the strategy game Go, it has led to self-driving cars, and it has led to machines that can play video games at a superhuman level.

Reinforcement learning has been around since the 70s but none of this has been possible until now.

The world is changing at a very fast pace. The state of California is changing their regulations so that self-driving car companies can test their cars without a human in the car to supervise.

We’ve seen that reinforcement learning is an entirely different kind of machine learning than supervised and unsupervised learning.

Supervised and unsupervised machine learning algorithms are for analyzing and making predictions about data, whereas reinforcement learning is about training an agent to interact with an environment and maximize its reward.

Unlike supervised and unsupervised learning algorithms, reinforcement learning agents have an impetus - they want to reach a goal.

This is such a fascinating perspective, it can even make supervised / unsupervised machine learning and "data science" seem boring in hindsight. Why train a neural network to learn about the data in a database, when you can train a neural network to interact with the real-world?

While deep reinforcement learning and AI has a lot of potential, it also carries with it huge risk.

Bill Gates and Elon Musk have made public statements about some of the risks that AI poses to economic stability and even our existence.

As we learned in my first reinforcement learning course, one of the main principles of training reinforcement learning agents is that there are unintended consequences when training an AI.

AIs don’t think like humans, and so they come up with novel and non-intuitive solutions to reach their goals, often in ways that surprise domain experts - humans who are the best at what they do.

OpenAI is a non-profit founded by Elon Musk, Sam Altman (Y Combinator), and others, in order to ensure that AI progresses in a way that is beneficial, rather than harmful.

Part of the motivation behind OpenAI is the existential risk that AI poses to humans. They believe that open collaboration is one of the keys to mitigating that risk.

One of the great things about OpenAI is that they have a platform called the OpenAI Gym, which we’ll be making heavy use of in this course.

It allows anyone, anywhere in the world, to train their reinforcement learning agents in standard environments.

In this course, we’ll build upon what we did in the last course by working with more complex environments, specifically, those provided by the OpenAI Gym:

  • CartPole

  • Mountain Car

  • Atari games

To train effective learning agents, we’ll need new techniques.

We’ll extend our knowledge of temporal difference learning by looking at the TD Lambda algorithm, we’ll look at a special type of neural network called the RBF network, we’ll look at the policy gradient method, and we’ll end the course by looking at Deep Q-Learning (DQN) and A3C (Asynchronous Advantage Actor-Critic).

Thanks for reading, and I’ll see you in class!


"If you can't implement it, you don't understand it"

  • Or as the great physicist Richard Feynman said: "What I cannot create, I do not understand".

  • My courses are the ONLY courses where you will learn how to implement machine learning algorithms from scratch

  • Other courses will teach you how to plug in your data into a library, but do you really need help with 3 lines of code?

  • After doing the same thing with 10 datasets, you realize you didn't learn 10 things. You learned 1 thing, and just repeated the same 3 lines of code 10 times...


Suggested Prerequisites:

  • College-level math is helpful (calculus, probability)

  • Object-oriented programming

  • Python coding: if/else, loops, lists, dicts, sets

  • Numpy coding: matrix and vector operations

  • Linear regression

  • Gradient descent

  • Know how to build ANNs and CNNs in Theano or TensorFlow

  • Markov Decision Proccesses (MDPs)

  • Know how to implement Dynamic Programming, Monte Carlo, and Temporal Difference Learning to solve MDPs


WHAT ORDER SHOULD I TAKE YOUR COURSES IN?:

  • Check out the lecture "Machine Learning and AI Prerequisite Roadmap" (available in the FAQ of any of my courses, including the free Numpy course)


UNIQUE FEATURES

  • Every line of code explained in detail - email me any time if you disagree

  • No wasted time "typing" on the keyboard like other courses - let's be honest, nobody can really write code worth learning about in just 20 minutes from scratch

  • Not afraid of university-level math - get important details about algorithms that other courses leave out

Content

Introduction and Logistics

Introduction and Outline
Where to get the Code
How to Succeed in this Course
Tensorflow or Theano - Your Choice!

Background Review

Review Intro
Review of Markov Decision Processes
Review of Dynamic Programming
Review of Monte Carlo Methods
Review of Temporal Difference Learning
Review of Approximation Methods for Reinforcement Learning
Review of Deep Learning

OpenAI Gym and Basic Reinforcement Learning Techniques

OpenAI Gym Tutorial
Random Search
Saving a Video
CartPole with Bins (Theory)
CartPole with Bins (Code)
RBF Neural Networks
RBF Networks with Mountain Car (Code)
RBF Networks with CartPole (Theory)
RBF Networks with CartPole (Code)
Theano Warmup
Tensorflow Warmup
Plugging in a Neural Network
OpenAI Gym Section Summary

TD Lambda

N-Step Methods
N-Step in Code
TD Lambda
TD Lambda in Code
TD Lambda Summary

Policy Gradients

Policy Gradient Methods
Policy Gradient in TensorFlow for CartPole
Policy Gradient in Theano for CartPole
Continuous Action Spaces
Mountain Car Continuous Specifics
Mountain Car Continuous Theano
Mountain Car Continuous Tensorflow
Mountain Car Continuous Tensorflow (v2)
Mountain Car Continuous Theano (v2)
Policy Gradient Section Summary

Deep Q-Learning

Deep Q-Learning Intro
Deep Q-Learning Techniques
Deep Q-Learning in Tensorflow for CartPole
Deep Q-Learning in Theano for CartPole
Additional Implementation Details for Atari
Pseudocode and Replay Memory
Deep Q-Learning in Tensorflow for Breakout
Deep Q-Learning in Theano for Breakout
Partially Observable MDPs
Deep Q-Learning Section Summary

A3C

A3C - Theory and Outline
A3C - Code pt 1 (Warmup)
A3C - Code pt 2
A3C - Code pt 3
A3C - Code pt 4
A3C - Section Summary
Course Summary

Theano and Tensorflow Basics Review

(Review) Theano Basics
(Review) Theano Neural Network in Code
(Review) Tensorflow Basics
(Review) Tensorflow Neural Network in Code

Appendix / FAQ

What is the Appendix?
Windows-Focused Environment Setup 2018
How to install Numpy, Scipy, Matplotlib, Pandas, IPython, Theano, and TensorFlow
How to Code by Yourself (part 1)
How to Code by Yourself (part 2)
How to Succeed in this Course (Long Version)
Is this for Beginners or Experts? Academic or Practical? Fast or slow-paced?
Proof that using Jupyter Notebook is the same as not using it
Python 2 vs Python 3
Is Theano Dead?
What order should I take your courses in? (part 1)
What order should I take your courses in? (part 2)
BONUS: Where to get Udemy coupons and FREE deep learning material

Screenshots

Advanced AI: Deep Reinforcement Learning in Python - Screenshot_01Advanced AI: Deep Reinforcement Learning in Python - Screenshot_02Advanced AI: Deep Reinforcement Learning in Python - Screenshot_03Advanced AI: Deep Reinforcement Learning in Python - Screenshot_04

Our review

📫 **Course Review Summary** **Introduction:** The online course on Deep Reinforcement Learning by The Lazy Programmer has garnered a high level of satisfaction among its recent participants. With an average global rating of 4.63, the course stands out for its thorough explanation of concepts and its practical approach, which is evidenced by the reviews shared below. **Pros:** - **Comprehensive Coverage:** The course provides a detailed exploration of reinforcement learning concepts, including common examples like Atari and cartpole. - **Practical Applications:** It offers concrete applications that help learners understand the practical use of the techniques taught. - **Clear Explanations:** The instructor takes the time to explain the rationale behind each technique and breaks down functions line by line, making complex topics more digestible. - **Structured Sections:** The course is divided into discrete sections, covering individual reinforcement learning algorithms, which facilitates understanding. - **Engaging Content:** Many reviewers found the course content engaging and enjoyable, with practical examples that complement the theoretical aspects. - **Deep Dive:** The course goes beyond surface-level explanations and delves into the mathematical details behind the algorithms, providing a comprehensive learning experience. - **Real-World Relevance:** Several participants mentioned that they learned more about deep learning and machine learning in these few hours than they did in college courses. - **Hands-On Learning:** The course encourages learners to practice and revise the material, which is crucial for understanding and retaining new information. - **Supportive Community:** Some reviewers pointed out that the community aspect of the course, with peers discussing and helping each other, was beneficial for learning. **Cons:** - **Outdated Code Examples:** A few reviews indicated that the code used in the course is outdated, referencing gym environments and libraries like Theano that may not align with current standards. - **Pacing Issues:** Some learners found the lecture speed to be a bit fast, making it challenging for beginners or those new to the subject to keep up. - **Unnecessary Content:** There were comments about the course including unnecessary technical advice outside of the core learning material, such as guidance on git, which some found distracting. - **Technical Updates Needed:** Reviewers suggested updating the exercises to use tensorflow-2 instead of tensorflow-1 and Theano to ensure compatibility with modern tools and practices. - **Lack of Hands-On Practice:** A few participants mentioned that while the theoretical aspects were well explained, they felt a need for more practical hands-on exercises to solidify their understanding. - **Instructional Style Critique:** Some learners criticized the instructor's method of delivering material, feeling it was more about how to learn rather than focusing on the content itself. **Final Verdict:** Overall, this course by The Lazy Programmer is a valuable resource for those looking to gain a deep understanding of deep reinforcement learning. Despite some criticisms regarding outdated code and pacing issues, the course's strengths in explaining concepts and providing practical examples make it a recommended learning path for individuals interested in advancing their skills in this field. **Recommendations:** - **Update Course Material:** To keep the course relevant and ensure that learners can apply what they've learned with current tools, updates to the code examples and exercises are recommended. - **Adjust Pacing:** Moderating the pace of lectures could help beginners follow along more easily and benefit from the material. - **Focus on Practical Application:** Encouraging more hands-on practice may enhance learners' ability to apply concepts in real-world scenarios. - **Incorporate Modern Tools:** Inclusion of exercises using tensorflow-2 would provide learners with the skills necessary for modern applications of reinforcement learning.

Coupons

DateDiscountStatus
5/16/201994% OFF
expired

Charts

Price

Advanced AI: Deep Reinforcement Learning in Python - Price chart

Rating

Advanced AI: Deep Reinforcement Learning in Python - Ratings chart

Enrollment distribution

Advanced AI: Deep Reinforcement Learning in Python - Distribution chart
1153742
udemy ID
3/22/2017
course created date
5/16/2019
course indexed date
Bot
course submited by