Build a Secure Data Lake in AWS using AWS Lake Formation

Step by step guide for setting up a data lake in AWS using Lake formation, Glue, DataBrew, Athena, Redshift, Macie etc.

4.35 (405 reviews)

Udemy

platform

English

language

Other

What you will learn

How to quickly setup a data lake in AWS using AWS Lake formation

You will learn to build real-world data pipeline using AWS glue studio and ingest data from sources such as RDS, Kinesis Firehose and DynamoDB

You will learn how to transform data using AWS Glue Studio and AWS Glue DataBrew

You will acquire good data engineering skills in AWS using AWS lake formation, Glue Studio and, blueprints and workflows in lake formation

Why take this course?

In this course, we will be creating a data lake using AWS Lake Formation and bring data warehouse capabilites to the data lake to form the lakehouse architecture using Amazon Redshift. Using Lake Formation, we also collect and catalog data from different data sources, move the data into our S3 data lake, and then clean and classify them.

The course will follow a logical progression of a real world project implementation with hands on experience of setting up a data lake, creating data pipelines for ingestion and transforming your data in preparation for analytics and reporting.

Chapter 1

Setup the data lake using lake formation
Create different data sources (MySQL RDS and Kinesis)
Ingest data from the MYSQL RDS data source into the data lake by setting up blueprint and workflow jobs in lake formation
Catalog our Database using crawlers
Use governed tables for managing access control and security
Query our data lake using Athena

Chapter 2,

Explore the use of AWS Gluw DataBrew for profiling and understanding our data before we starting performing complex ETL jobs.
Create Recipes for manipulating the data in our data lake using different transformations
Clean and normalise data
Run jobs to apply the recipes on all new data or larger datasets

Chapter 3

Introduce Glue Studio
Author and monitor ETL jobs for tranforming our data and moving them between different zone of our data lake
Create a DynamoDB source and ingest data into our data lake using AWS Glue

Chapter 4

Introduce and create a redshift cluster to bring datawarehouse capabilities to our data lake to form the lakehouse architecture
Create ETL jobs for moving data from our lake into the warehouse for analytics
Use redshift spectrum to query against data in our S3 data lake without the need for duplicating data or infrastructure

Chapter 5

Introduce Amazon Macie for managing data security and data privacy and ensure we can continue to identify sensitive data at scale as our data lake grows

Screenshots

Build a Secure Data Lake in AWS using AWS Lake Formation - Screenshot_01

Build a Secure Data Lake in AWS using AWS Lake Formation - Screenshot_02

Build a Secure Data Lake in AWS using AWS Lake Formation - Screenshot_03

Build a Secure Data Lake in AWS using AWS Lake Formation - Screenshot_04

Reviews

Jongha

September 16, 2023

Perfect things in this lecture what I want to learn. !!!!! Excellent. Why don't you make the lectures for stream and batch for lambda !!!

Ricardo

September 15, 2023

The content of the course is good and it is oriented to those people who look for a step-by-step demo to use AWS tools involved in a data lake solution. However, I will hardly buy another course from this author. His accent makes very difficult to follow the explanations, and I have a case in lesson 6 section 2, where the assignment of permissions in Lake Formation did not work. Unfortunately, the presenter omits several details that might be important to make this configuration, e.g. permissions needed in users or roles in IAM.

JULIO

July 10, 2023

was great!! just want more practical exercises and more focuse on lakeformation and iceberg , but great intro course

Jose

June 30, 2023

The course offers excellent content, although the instructor's accent can pose a slight challenge to understanding.

Hafiz

June 25, 2023

Course is good enough but it is lacking more of lackformation specific features like governance, authentication with SAML etc. More focused should have been on Governance

Michel

April 21, 2023

All the IAM Policies and Roles are poorly presented, if they are not missing at all. Got stuck way to often just to figure out the correct policies and lost a lot of time.

Eghosa

January 30, 2023

The learning experience from the basics to the crux of data lake formation has been very fascinating and interesting.

Christian

January 24, 2023

For me it was too much detail, I'd prefer the focus to be on understanding how the services work. This was very much a step-by-step walkthrough of how to do it. I kind of lost track of the overall picture in all the details.

Samuel

January 13, 2023

It's hard to rate this course. On one hand it's exactly what I was looking for at first, on the other hand it left me with so many things wanting. This is basically a technical demo of Lake Formation, Glue, Redshift and Macie in the context of building your own data lake. However, you should not expect this course will give you everything you will need to start building a data lake yourself. It merely gives you an idea of what is possible in AWS and how complex it may be. But the course doesn't explain different basic concepts and doesn't go deep enough. You also shouldn't be an AWS beginner but should already have basic knowledge into at least IAM and S3.

David

January 13, 2023

Good hands-on demo. It would be helpful to include transcript of the many steps involved. I ended up creating my own step by step process. It would also be helpful to have slides showing which IAM users, groups, roles, policies and EC2 security groups are required for each step. The only download is a customer.csv. Does not include: slides, transcripts, cheatsheets, or quizzes.

Veronica

January 10, 2023

It shows detail demonstrations on how to create a data lake in a simple way. But if you require more details on the tools used is needed to compliment it with additional resources

Carlos

November 13, 2022

awesome course! overall content was very well thought out! many demos. sometimes we get lost with so many iam roles but i could finish it! congratulations and keep up!!

Joseph

November 8, 2022

Good demonstration of what to do, but little explanation of why to do certain things while going through the steps. As we progress, it may be more clear why he did what he did. Also, lots of "uh" and "ah". It's great as a quick overview of what can be done, but don't expect to really understand how or why to do everything if you are a beginner. You'll have to set up your own environment and go through things step by step on your own trying to replicate what you see.

Fabio

October 6, 2022

Overall overview of AWS tooling was good. The on-hands training is lacking of organization, naming conventions, proper data sets which ins't ideal for learning process. Also, the architecture initial design was not fulfilled with curated bucket being empty at the end.

Naga

September 25, 2022

Didn't followed the order of the flow chart which is shown in this syllabus. Some vidoes parts need to be updated like "Creating Connections" for RDS . Now in console Add Connections options itslef is not available it is very different now.

Charts

Price

Rating

Enrollment distribution

Build a Secure Data Lake in AWS using AWS Lake Formation - Distribution chart