Modern Web Scraping Fundamentals with Python

Harness the power of Scrapy, BeautifulSoup and Selenium to boost your webscraping game!

4.25 (786 reviews)
Udemy
platform
English
language
Other
category
instructor
34,191
students
3.5 hours
content
Mar 2024
last update
$74.99
regular price

What you will learn

Understand the most important components for web scraping

Build their own web scraping projects

Learn core components of two of the most powerful scraping libraries: Scrapy and BeautifulSoup

Build multiple hands-on projects

Description

Getting access to the data you need can make or break you.

This is why Fortune 500 companies like Walmart, CNN, Target, and HSBC use web scraping to get ahead and stay ahead with data.

It’s the original growth tool and one of their best-kept secrets


…And it can easily be yours too.


From data spoofing to legalities, crawling libraries, maintenance, monitoring, more, building a safe and effective web scraper is risky business but it’s a skill every data scientist needs in their toolkit.


Today, we’re building one from scratch.


Hi, my name is Jordan Sauchuk. I’m an AI & Cybersecurity Engineer and a SuperDataScience instructor. I’m here to guide you step-by-step in building custom web scrapers in Python using Selenium, Scrapy and BeautifulSoup.


Welcome to Modern Web Scraping in Python.


At the end of this course, you will understand the most important components of web scraping and be able to build your own web scrapers to obtain new data, optimize internal processes and more.


Plus, familiarize yourself with some of the most common scraping techniques and sharpen your Python programming skills while you’re at it!


  1. First, learn the essentials of web scraping, explore the framework of a website and get your local environment ready to take on scraping challenges with Scrapy, BeautifulSoup, and Selenium.

  2. Next, set up a Scrapy crawler and cover the core details that can be applied to building datasets or mining.

  3. Next, cover the basics of BeautifulSoup, utilize the requests library and LXML parser, and scale up to deploy a new scraping algorithm to scrape top product information from Amazon

  4. Fourth, set up Selenium, and deploy it to solve a practical, real-world challenge. Plus, submit your solution to obtain useful feedback from me.

  5. Finally, test your newfound skills on a cybersecurity project that involves you finding highly-sensitive data.


We’l bel coding in Python, and using the automated testing suite Selenium, the Python framework Scrapy, and library BeautifulSoup to build web scrapers that can be customized to your specific needs.


But a thorough walk-through isn’t all you’re getting.


Access our student forum where you can interact with me and your fellow students. Ask me questions, receive input from other students and be inspired by the clever scraping solutions from your classmates.


Whether you’re a data scientist, machine learning or AI engineer who wants to access more data sources; a web developer looking to automate tasks, or a data buff with a general interested in data science and web scraping…


This course delivers an in-depth presentation of web scraping basics, methodologies and approaches that you can easily apply to your own personal projects, or out there in the real world of business.


Join me now and let’s start scraping the web together. Enroll today.


Content

It's super easy to get started

Welcome!
Simple HTTP Request
What is a website + the DOM
Sublime
The Inspector
Traversing The Tree
Robots.txt
Anaconda Update Notice
Anaconda Download
Setting up your virtual environment
Changes
Get The Files Here!
Your Shortcut To Becoming A Better Data Scientist

Practical 1 - Scrapy Challenge

Introduction and Installing Scrapy
Starting your Scrapy Project
Building your first Spider
Scrapy Shell
Selectors
Scrapy Items
Request + Response Quick
Traversing Options
Challenge Intro
Challenge Solution
Scrapy Cloud
Scrapy Recap Quiz

Practical 2 - BeautifulSoup Challenge

BS Installation
BS Requests
BS Basics
BS Basics cont.
BS Challenge Intro
BS Challenge Solution
BS Recap Quiz

Practical 3 - Selenium + BeautifulSoup

GitHub
Section Intro + Selenium Installation
Resources for Selenium
Selenium - Click Revised
Selenium Login Keys
Selenium Challenge Intro
Recommended: Build Your Own Approach First
Selenium Challenge Pt. 1
Selenium Challenge Solution
Selenium Project Feedback

Course Challenge

Introduction
Recommended: Build Your Own Approach First
Solution Pt. 1
Solution Pt. 2
Github Submission

Screenshots

Modern Web Scraping Fundamentals with Python - Screenshot_01Modern Web Scraping Fundamentals with Python - Screenshot_02Modern Web Scraping Fundamentals with Python - Screenshot_03Modern Web Scraping Fundamentals with Python - Screenshot_04

Reviews

Andreas
July 21, 2023
Just seems a bit disconnected and explanations seem to assume various levels of expertise depending where in the lectures you are. Also when listening through headphones the popping of the Mike seems a little annoying. I would expect later on hopefully this lecture flows a bit better.
Valentin
April 17, 2023
As a beginner in programming I struggle to pass the first steps of the installation process (I think like most people who don't already know programming) The teacher speaks too fast and the approach is a bit too theoretical. I recommend you don't take this course. I think datacamp is better. but at least it is refundable
Soffan
March 5, 2023
I have watched a quarter of the videos.. but haven't tried myself. The explanation is straight to the point and well understood. Hope it will be easy when practicing.
Fernando
August 30, 2022
Good for people starting out, not really what I was expecting (a more in-depth look into scrapy, unfortunately this is too surface-level for my job needs).
Hristo
February 6, 2022
If you have no idea about web scrapping that course is probably for you. It is really really basic. Except for a few things I did not learn anything valuable. And font size is small and you can not see the mentor's code.
Vladimir
October 28, 2021
he aprendido bastante del tema. Es necesario ir paso a paso para poder aprender a usar las herramientas.
Benjamin
October 25, 2021
The course is a good introduction, but moves forward way to fast and misses details. I would expect an overarching structure that mentions that the beginning how web-scraping is used and then explain later the main use cases. For example, fetching three books listed on a website is good for a start, but one needs more complex projects to learn and get into the details. This is missing here.
Qayad
October 4, 2021
A very good introduction to web scrapping using Scrapy and BeautifulSoup. Adding Selenium to the mix just took these tools to a whole other level. Nice job!!!
Aflah
August 29, 2021
what I like the most here and make this course so much better than other courses is the instructors who are passionate to teach this can be seen from the frequent responses to QnA and the many challenges that are given so that it gives many opportunities to apply what is taught
Jay
August 11, 2021
I don't understand why a lot of teachers go through all the work of producing their course(s) and yet overlook one of the 2 most important factors. "Making the screen text large enough to see well". I mean a ton of Udemy teachers are guilty of this. SMH
Vishnubabu
August 9, 2021
I learned a decent amount of scraping but Jordan is a lot messy while working on concepts and explaining them.
Karl
July 28, 2021
Having the bonus content at the beginning of the course was confusing - recommend moving it to the last section.
Yu
July 21, 2021
I have only done the Scrapy section so far, but it seems pretty clear that the course will not be sufficient for me to learn web scraping for real. It's more like a quick live demo of the libraries documentation to get your hands dirty. Maybe that's the goal of this course? Unfortunately, the instructor don't really explain the core fundamental concepts and use a lot of cut-corner techniques that end up getting the students in trouble. For example, content selection with XPath and CSS is not really well explained, the author just point you to the documentation and then copy paste some selection code from Chrome developer console. This normally would work, until it doesn't and you don't have the knowledge to debug or to develop a work around. Quite a few students run into this problem in the Q&A and got stuck. Overall, the course so far seems like it might be a good guided exercise, but with so many things left out and jumping all over the place, I don't think I can benefit from finishing it. Even though I got this course for free, I ended up purchasing another course with basically the same name and learned a lot more from it within an hour. I guess thanks to this course leading me to mess around with Scrapy trying to get it work!
Mohammad
July 2, 2021
The video quality much more better to understand, because the code in the video is not clear. The content is good.
Anvinder
June 27, 2021
THis course has been a waste of my time. - Uptil 20 lectures there was nothing important at all. It's like the person has added all these lectures just to increase the course duration. I still dont understand why has the instructor tried to make it difficult to learn scrapy by doing all the stuff with the terminal etc. Why not just use 1 IDE and let the people learn. Had to buy another course.

Coupons

DateDiscountStatus
11/29/2020100% OFF
expired
3/12/2021100% OFF
expired

Charts

Price

Modern Web Scraping Fundamentals with Python - Price chart

Rating

Modern Web Scraping Fundamentals with Python - Ratings chart

Enrollment distribution

Modern Web Scraping Fundamentals with Python - Distribution chart

Related Topics

3331104
udemy ID
7/14/2020
course created date
8/24/2020
course indexed date
Bot
course submited by