Modern Reinforcement Learning: Deep Q Agents (PyTorch & TF2)

How to Turn Deep Reinforcement Learning Research Papers Into Agents That Beat Classic Atari Games

4.46 (1028 reviews)
Udemy
platform
English
language
Data Science
category
instructor
Modern Reinforcement Learning: Deep Q Agents (PyTorch & TF2)
5,970
students
7 hours
content
Aug 2023
last update
$79.99
regular price

What you will learn

How to read and implement deep reinforcement learning papers

How to code Deep Q learning agents

How to Code Double Deep Q Learning Agents

How to Code Dueling Deep Q and Dueling Double Deep Q Learning Agents

How to write modular and extensible deep reinforcement learning software

How to automate hyperparameter tuning with command line arguments

Why take this course?

In this complete deep reinforcement learning course you will learn a repeatable framework for reading and implementing deep reinforcement learning research papers. You will read the original papers that introduced the Deep Q learning, Double Deep Q learning, and Dueling Deep Q learning algorithms. You will then learn how to implement these in pythonic and concise PyTorch and Tensorflow 2 code, that can be extended to include any future deep Q learning algorithms. These algorithms will be used to solve a variety of environments from the Open AI gym's Atari library, including Pong, Breakout, and Bankheist.


You will learn the key to making these Deep Q Learning algorithms work, which is how to modify the Open AI Gym's Atari library to meet the specifications of the original Deep Q Learning papers. You will learn how to:

  • Repeat actions to reduce computational overhead

  • Rescale the Atari screen images to increase efficiency

  • Stack frames to give the Deep Q agent a sense of motion

  • Evaluate the Deep Q agent's performance with random no-ops to deal with model over training

  • Clip rewards to enable the Deep Q learning agent to generalize across Atari games with different score scales


If you do not have prior experience in reinforcement or deep reinforcement learning, that's no problem. Included in the course is a complete and concise course on the fundamentals of reinforcement learning. The introductory course in reinforcement learning will be taught in the context of solving the Frozen Lake environment from the Open AI Gym.

We will cover:

  • Markov decision processes

  • Temporal difference learning

  • The original Q learning algorithm

  • How to solve the Bellman equation

  • Value functions and action value functions

  • Model free vs. model based reinforcement learning

  • Solutions to the explore-exploit dilemma, including optimistic initial values and epsilon-greedy action selection

Also included is a mini course in deep learning using the PyTorch framework. This is geared for students who are familiar with the basic concepts of deep learning, but not the specifics, or those who are comfortable with deep learning in another framework, such as Tensorflow or Keras. You will learn how to code a deep neural network in Pytorch as well as how convolutional neural networks function. This will be put to use in implementing a naive Deep Q learning agent to solve the Cartpole problem from the Open AI gym. 

Screenshots

Modern Reinforcement Learning: Deep Q Agents (PyTorch & TF2) - Screenshot_01Modern Reinforcement Learning: Deep Q Agents (PyTorch & TF2) - Screenshot_02Modern Reinforcement Learning: Deep Q Agents (PyTorch & TF2) - Screenshot_03Modern Reinforcement Learning: Deep Q Agents (PyTorch & TF2) - Screenshot_04

Reviews

Steven
April 21, 2023
Great material, especially the breaks where he expects you to attempt to code up the material on your own instead of spoon feeding it and the lessons on converting papers to code.
Stephane
April 9, 2023
si vous voulez apprendre la théorie du deep learning reinforcement basé sur le qlearning, ce formateur vous resume la théorie mathématique de facon simplifiée et accessible, et il vous incite a coder les algos tout en vous montrant les bonnes pratiques. extraordinaire formation !
Marcos
March 1, 2023
Rely too much on the gym environment. It doesn't teach very well how to implement your own environment based on a new problem as I expected.
Ranjeet
December 12, 2022
try to explain the meaning and concept of each code syntax and block..like if --main--name ,flag=done etc,how these things work in code
Ahmad
September 15, 2022
Please give a little more time on the background of the RL. Epsilon Greedy and Bellman equation + its variations. Can not we have an implementation in TensorFlow code?
Yacine
August 11, 2022
Everything's *excellent* and I've truely learned a lot. Here's something though : felt like there was a lack of "mechanical"/mathematical details on exp replay, double and dueling networks. Examples: Exp replay reduces correlations between what is fed to the neural net, but how *exactly* are the experiences correlated ? intuitively speaking it's some kind of path dependency. Double networks : the mathematical explanation is skipped, which would be alright if there was a somewhat intuitive way to understand why there can be overestimations.
Elidor
July 1, 2022
It was a good course that also focused a lot on good coding practices! The methodology used to explain how to implement research papers was really good and it represents a useful skill that can be used in real-life projects. The only drawback is that it has a harder than expected learning curve at least for the beginning of the course when everything is being started from scratch, but the author takes care of that by also providing a useful Github repository that can be used as a comparison between the student code and the one explained during the course lessons. Would totally recommend it to both students and industry professionals!
Jorge
June 1, 2022
The course is relying on the gym environment too much. For those who need their own environment it is kind of stress and anxiety hoping to get there in the next lessons, I hope
Ahmed
February 20, 2022
Well-organized thoughts and very good way of delivering information. I like the instructor's style of programming. The introduction to Q-learning is good, however reading the equations in each slide as they are (e.g. G sub t equals ...) is confusing. I understand that all symbols were properly introduced at first, but I prefer always pronouncing equations in terms of the actual meaning rather than symbols.
Kwan-Yuet
February 6, 2022
As someone who knows a bit of the topic, it is a good recap. For those who is a starter, they might not have a clue.
Ajay
December 28, 2021
Hi Phil, I have gone through you two course (M RL Actor-Critic and M RL Deep Q Learning) to solve my problem for pricing optimization on Retail but I am unable to kick start my journey. Do you have any other course where I can learn to work on Offline RL to build Conservative Q-Learning  model. I Request you to please give your feedback on solving this problem or any suggestions will be great help. Regards, Ajay Kumar
Constantine
October 21, 2021
It was a great, solid, in depth course that finally allowed me to understand the "Deep" in Deep RL, and not only that, he also focused on writing good quality code which is yet another very important part of the whole process. Can't wait to get started on his Policy Gradient course!
Fernando
September 30, 2021
After a while, the complexity increases and the quality of "teaching" decreases... Very disappointing really, the distribution of the course leaves subjects pending so you have to keep watching videos. While you wait for the author to resume the subject, it is already late. What you were waiting for is not explained and you can't refund the course. Example: Section 3... 55 minutes explaining DQL for continuous state spaces and the code doesn't have the implementation of memory, he ends the sections showing bad performance in his code(even he says it). Then in Section 4... He rectifies his own code, saying that the ways implemented before were wrong..... Meanwhile he's coding, he does not explain how this code fit into the theory of the algorithm... Summary: all the explanation is superficial, it's much better if you just check the code of his github and google the subjects or parts of the code, because everything in this course you can get it for free somewhere else. I've seen a lot of videos on youtube with the same or even better quality content. If you are looking for a great explanatory step-by-step code with a nice gradually increase in complexity: go somewhere else, this is not the course. The idea of paying for a course is that the author MUST give you knowledge(quality or insides) that you can't get it for free somewhere else.
Roy
July 26, 2021
One of the best courses I did online. It is comprehensive, clear and to the point. Starting this course I had solid understanding in DNN but didn't know any RL, I feel this has changed. Cheers.
Joachim
June 22, 2021
It would be great if the paper would be parallel discussed to the code when implementing the solutions

Charts

Price

Modern Reinforcement Learning: Deep Q Agents (PyTorch & TF2) - Price chart

Rating

Modern Reinforcement Learning: Deep Q Agents (PyTorch & TF2) - Ratings chart

Enrollment distribution

Modern Reinforcement Learning: Deep Q Agents (PyTorch & TF2) - Distribution chart

Coupons

DateDiscountStatus
3/14/202192% OFF
expired

Related Topics

2662326
udemy ID
11/19/2019
course created date
1/7/2020
course indexed date
Bot
course submited by