Software Engineering


Autonomous Cars: Deep Learning and Computer Vision in Python

Learn OpenCV, Keras, object and lane detection, and traffic sign classification for self-driving cars

4.74 (841 reviews)


13 hours


Apr 2021

Last Update
Regular Price

What you will learn

Automatically detect lane markings in images

Detect cars and pedestrians using a trained classifier and with SVM

Classify traffic signs using Convolutional Neural Networks

Identify other vehicles in images using template matching

Build deep neural networks with Tensorflow and Keras

Analyze and visualize data with Numpy, Pandas, Matplotlib, and Seaborn

Process image data using OpenCV

Calibrate cameras in Python, correcting for distortion

Sharpen and blur images with convolution

Detect edges in images with Sobel, Laplace, and Canny

Transform images through translation, rotation, resizing, and perspective transform

Extract image features with HOG

Detect object corners with Harris

Classify data with machine learning techniques including regression, decision trees, Naive Bayes, and SVM

Classify data with artificial neural networks and deep learning


Autonomous Cars: Computer Vision and Deep Learning

The automotive industry is experiencing a paradigm shift from conventional, human-driven vehicles into self-driving, artificial intelligence-powered vehicles. Self-driving vehicles offer a safe, efficient, and cost effective solution that will dramatically redefine the future of human mobility. Self-driving cars are expected to save over half a million lives and generate enormous economic opportunities in excess of $1 trillion dollars by 2035. The automotive industry is on a billion-dollar quest to deploy the most technologically advanced vehicles on the road.

As the world advances towards a driverless future, the need for experienced engineers and researchers in this emerging new field has never been more crucial.

The purpose of this course is to provide students with knowledge of key aspects of design and development of self-driving vehicles. The course provides students with practical experience in various self-driving vehicles concepts such as machine learning and computer vision. Concepts such as lane detection, traffic sign classification, vehicle/object detection, artificial intelligence, and deep learning will be presented. The course is targeted towards students wanting to gain a fundamental understanding of self-driving vehicles control. Basic knowledge of programming is recommended. However, these topics will be extensively covered during early course lectures; therefore, the course has no prerequisites, and is open to any student with basic programming knowledge. Students who enroll in this self-driving car course will master driverless car technologies that are going to reshape the future of transportation.

Tools and algorithms we'll cover include:

  • OpenCV

  • Deep Learning and Artificial Neural Networks

  • Convolutional Neural Networks

  • Template matching

  • HOG feature extraction


  • Tensorflow and Keras

  • Linear regression and logistic regression

  • Decision Trees

  • Support Vector Machines

  • Naive Bayes

Your instructors are Dr. Ryan Ahmed with a PhD in engineering focusing on electric vehicle control systems, and Frank Kane, who spent 9 years at Amazon specializing in machine learning. Together, Frank and Dr. Ahmed have taught over 200,000 students around the world on Udemy alone.

Students of our popular course, "Data Science, Deep Learning, and Machine Learning with Python" may find some of the topics to be a review of what was covered there, seen through the lens of self-driving cars. But, most of the course focuses on topics we've never covered before, specific to computer vision techniques used in autonomous vehicles. There are plenty of new, valuable skills to be learned here!


Autonomous Cars: Deep Learning and Computer Vision in Python
Autonomous Cars: Deep Learning and Computer Vision in Python
Autonomous Cars: Deep Learning and Computer Vision in Python
Autonomous Cars: Deep Learning and Computer Vision in Python


Environment Setup and Installation


Installation Notes: OpenCV3 and Python 3.7

Install Anaconda, OpenCV, Tensorflow, and the Course Materials

Test your Environment with Real-Time Edge Detection in a Jupyter Notebook

Udemy 101: Getting the Most From This Course

Introduction to Self-Driving Cars

A Brief History of Autonomous Vehicles

Course Overview and Learning Outcomes

Python Crash Course [Optional]

Python Basics: Whitespace, Imports, and Lists

Python Basics: Tuples and Dictionaries

Python Basics: Functions and Boolean Operations

Python Basics: Looping and an Exercise

Introduction to Pandas

Introduction to MatPlotLib

Introduction to Seaborn

Computer Vision Basics: Part 1

What is computer vision and why is it important?

Humans vs. Computers Vision system

what is an image and how is it digitally stored?

[Activity] View colored image and convert RGB to Gray

[Activity] Detect lane lines in gray scale image

[Activity] Detect lane lines in colored image

What are the challenges of color selection technique?

Color Spaces

[Activity] Convert RGB to HSV color spaces and merge/split channels

Convolutions - Sharpening and Blurring

[Activity] Convolutions - Sharpening and Blurring

Edge Detection and Gradient Calculations (Sobel, Laplace and Canny)

[Activity] Edge Detection and Gradient Calculations (Sobel, Laplace and Canny)

[Activity] Project #1: Canny Sobel and Laplace Edge Detection using Webcam

Computer Vision Basics: Part 2

Image Transformation - Rotations, Translation and Resizing

[Activity] Code to perform rotation, translation and resizing

Image Transformations – Perspective transform

[Activity] Perform non-affine image transformation on a traffic sign image

Image cropping dilation and erosion

[Activity] Code to perform Image cropping dilation and erosion

Region of interest masking

[Activity] Code to define the region of interest

Hough transform theory

[Activity] Hough transform – practical example in python

Project Solution: Hough transform to detect lane lines in an image

Computer Vision Basics: Part 3

Image Features and their importance for object detection

[Activity] Find a truck in an image manually!

Template Matching - Find a Truck

[Activity] Project Solution: Find a Truck Using Template Matching

Corner detection – Harris

[Activity] Code to perform corner detection

Image Scaling – Pyramiding up/down

[Activity] Code to perform Image pyramiding

Histogram of colors

[Activity] Code to obtain color histogram

Histogram of Oriented Gradients (HOG)

[Activity] Code to perform HOG Feature extraction

Feature Extraction - SIFT, SURF, FAST and ORB

[Activity] FAST/ORB Feature Extraction in OpenCV

Machine Learning: Part 1

What is Machine Learning?

Evaluating Machine Learning Systems with Cross-Validation

Linear Regression

[Activity] Linear Regression in Action

Logistic Regression

[Activity] Logistic Regression In Action

Decision Trees and Random Forests

[Activity] Decision Trees In Action

Machine Learning: Part 2

Bayes Theorem and Naive Bayes

[Activity] Naive Bayes in Action

Support Vector Machines (SVM) and Support Vector Classifiers (SVC)

[Activity] Support Vector Classifiers in Action

Project Solution: Detecting Cars Using SVM - Part #1

[Activity] Detecting Cars Using SVM - Part #2

[Activity] Project Solution: Detecting Cars Using SVM - Part #3

Artificial Neural Networks

Introduction: What are Artificial Neural Networks and how do they learn?

Single Neuron Perceptron Model

Activation Functions

ANN Training and dataset split

Practical Example - Vehicle Speed Determination

Code to build a perceptron for binary classification

Backpropagation Training

Code to Train a perceptron for binary classification

Two and Multi-layer Perceptron ANN

Example 1 - Build Multi-layer perceptron for binary classification

Example 2 - Build Multi-layer perceptron for binary classification

Deep Learning and Tensorflow: Part 1

Intro to Deep Learning and Tensorflow

Building Deep Neural Networks with Keras, Normalization, and One-Hot Encoding.

[Activity] Building a Logistic Classifier with Deep Learning and Keras

ReLU Activation, and Preventing Overfitting with Dropout Regularlization

[Activity] Improving our Classifier with Dropout Regularization

Deep Learning and Tensorflow: Part 2

Convolutional Neural Networks (CNN's)

Implementing CNN's in Keras

[Activity] Classifying Images with a Simple CNN, Part 1

[Activity] Classifying Images with a Simple CNN, Part 2

Max Pooling

[Activity] Improving our CNN's Topology and with Max Pooling

[Activity] Build a CNN to Classify Traffic Signs

[Activity] Build a CNN to Classify Traffic Siigns - part 2

Wrapping Up

Bonus Lecture: More courses to explore!


Mike30 August 2020

Frank provides great concise explanations of material. But the sections covered by Dr. Ryan were just okay - the demonstrations were good, but I don't think he's great at teaching.

Satish11 August 2020

It is a really good course for beginners to Deep Learning and Autonomous Vehicles. Concepts are very well explained. Ryan is very descriptive but a little slow at times, but puts his point across very effectively.

Muhammad5 July 2020

If there is any changes it should be reflected in the lecture video as well or put some additional notes so that further inconvenience should be avoided. For example In lecture 4.4 there is a change np.set_printoptions(threshold=np.nan) and It took 30 min for me to get back to the lecture with solution so that i will move forward in the lecture. Thus, I request to improve the current way.

Alex17 June 2020

If you want to grab a USB camera and make a stand alone vision setup for lab use, this class may not be for you. This is a basic class where you will need to sweat out the Windows/pathing/VM issues to make a desktop vision system, that's what this is. If you want a stand alone unit that detects birds in your yard, this is not it.

Aditya16 June 2020

I enjoyed the course. However, if you don't have any idea about programming then it will be tough as they don't go much into the depth of why we used this or that code. I found Ryan to be good as he explained everything in detail. I will likely to be buying more courses of him. Moreover, this course doesn't include taking existing deep learning model off the web and modifying it. Data used are mainly images and not shown how we can work through the video feeds. Would be lot improved if they had added couple of live video feed deep learning model and how to do it. Overall good course though.

Eric29 January 2020

Overall, a good summary overview of the material. The lectures are succinct and the alternating of “lecture” and “activity” keep the material interesting and moves the content along. Further references to a more theoretical review of the material would be a welcome addition. Also, the audio skips and seems to misalign with the slides at times. This was the worst during the feature detection (SIFT, SURF, etc.) lecture where the slides shown were on HOG. This could be associated with using the Udemy App on an iPad only...I did not verify.

Michael28 January 2020

The course information is solid and I'm coming away with a much better understanding of python, Neural Networks, Deep Learning, image recognition and the tools that are used for Autonomous cars. However, the course material may need some updates. There were a few instances where the Jupyter code samples didn't run. I took the extra step to investigate most of the issues and correct it so that they would run properly in my code, but other viewers may not do this. Also, towards the end, I believe Section 10, one of the final activity videos was replaying previous course videos. It seems like there was an error in the video upload process. I commented in the section for correction.

David24 January 2020

This course will expose you to many interesting technologies, such as openCV, and the instructors are excellent. You should have a good understanding of python when the slicing and dicing starts. As well as updating conda packages and coding tools. I take online courses at work and at home, I sometimes stop for a while then come back to it. This means that I have to “switch gears” in my head to remember the technologies, so I like a quick recap sort of like how t.v. series start with “last time on …”. The instructors did take time to that. I have a current valid C.D.L. an have paid working experience as a truck driver and bus driver in my past. Understanding more of how self-driving algorithms and technologies are going to work is surely and interesting topic, but as a driver it is a bit concerning.

Bryce3 December 2019

The course title is a bit misleading. It covers little about autonomous cars. This is a very basic/entry level machine learning and deep learning course.

Stanislav9 November 2019

The installation procedure is a hell. Struggling for 5 hours so far. Why don't you install not the latest but specific packages of everything?

Vamsinag19 October 2019

The videos are not updated with the latest tool installation details. I have spent more than a month just to try and get the environments working but in vain and the support available from the teaching assistants is absolutely meager! It would be helpful if the moderators themselves tried to install all the tools freshly and perhaps document all the problems faced and the solutions that helped.

Niko29 September 2019

Well presented and to the point information. Some difficulty setting up the environment, but the Q&A section usually has an answer to my problems.

Ehsan21 September 2019

I was, as expected for a basic course. The course doesn't cover some important concepts such as object detection and localization

Mustafa9 September 2019

it was the greate course which is prepared diligently. I learn too many details and examples from this class Thank you for all

Ron7 September 2019

Sun-dog purses with Frank Kane are always thorough and easy to follow. His explanations are in-depth enough to really understand the material.


Udemy ID


Course created date


Course Indexed date
Course Submitted by