Databricks Essentials for Spark Developers (Azure and AWS)

Learn about Databricks Features

4.25 (955 reviews)
Udemy
platform
English
language
Other
category
Databricks Essentials for Spark Developers (Azure and AWS)
13,167
students
4 hours
content
Sep 2019
last update
$44.99
regular price

What you will learn

Using Community Edition of Databricks to explore the platform

Signing up for Full Trial using Azure Databricks

Signing up for Full Trial using Databricks on AWS

Develop and Deploy Notebooks using Scala, Python as well as SQL using Databricks Platform

Understand the difference between interactive and job clusters

Formal Development and Deployment Life Cycle

Run jobs by attaching application as jar along with libraries

Overview of Cluster Pools

Installing and using databricks-cli

Description

Are you an experienced Spark Developer and want to understand about Databricks platform?

Conventionally we used to work on Data Engineering with clusters built using distributions. However, with cloud if we can separate storage from compute and leverage pay as you go model, the costs of infrastructure for Big Data Clusters can go down significantly.

Databricks is one such Cloud Choice!!!

As part of this course, you will be learning the essentials of Databricks Essentials.

  • Understand different editions such as Community, Databricks (AWS) and Azure Databricks

  • Signing up for community edition

  • Uploading data to DBFS

  • Developing using Databricks Notebook with Scala, Python as well as Spark SQL

  • Development Life Cycle using Scala with IntelliJ as IDE

  • Configuring jobs using Jar files

  • and many more

Content

Getting Started with Databricks

Introduction to the course
Signing up for Databricks Community Edition
Create Azure Databricks Service
Signup For Databricks Full Trial on AWS
Overview of Databricks UI
Upload Files into Databricks
Create Cluster in Databricks Platform
Managing File System using Notebooks
Using Databricks UI

Databricks Notebook using Scala with Spark

Define Problem Statement
Create Notebook and read orders using Scala
Read order_items and products into DataFrames using Scala
Visualize Data using Display in Scala
Get complete or closed orders using Scala
Join and select data using Scala
Get Daily Product Revenue using Scala
Get Daily Product Revenue Sorted using Scala
Save output to CSV Files using Scala
Exercise - Get the monthly revenue for each Customer using Scala

Databricks Notebook using Python (pyspark)

Define Problem Statement
Create Notebook and read orders using Python
Read order_items and products into DataFrames using Python
Visualize Data using Display in Python
Get complete or closed orders using Python
Join and select data using Python
Get Daily Product Revenue using Python
Get Daily Product Revenue Sorted using Python
Save output to CSV Files using Python
Exercise - Get the monthly revenue for each Customer using Python

Databricks Notebook using Spark SQL

Create Notebook and read orders using Spark SQL
Read order_items and products using Spark SQL
Visualize SQL results
Join and Select data using Spark SQL
Get Daily Product Revenue Sorted using Spark SQL
Save output to a table using Spark SQL
Exercise - Get the monthly revenue for each Customer using Spark SQL

Databricks Jobs and Clusters

Introduction to Jobs and Cluster
Create Cluster on Azure Databricks
Request to increase CPU Quota on Azure
Creating Job on Databricks using Notebook
Submitting Jobs using Job Cluster
Create Pool in Databricks
Running Job using Interactive Cluster attached to Pool
Running Job using Job Cluster attached to Pool
Exercise - Submit the application as job using interactive cluster

Databricks Development and Deployment Life Cycle using Scala

Introduction
Setup Project using IntelliJ IDE
Add Dependencies to the project
Develop application using IDE
Externalize Properties
Submit job using Jar

Reviews

Gnanaprakasam
January 29, 2023
This is an excellent course to understand databricks with Azure and AWS environments to submit spark jobs. All Durga courses are hands-on and easy to understand. I strongly recommend this course to anyone who wants to understand Databricks running on cloud.
JK
January 21, 2023
There is significant difficulty just in grasping many concepts within IT. Having to 'work' to understand instructors communication only makes it harder to learn. If courses are to be conducted in English for primarily an English-speaking audience then expecting a greater level of diction and pronunciation should be a prerequisite. No doubt the instructor knows the material but communicating that clearly and concisely is another matter.
Isaac
November 3, 2022
Good teaching by the Instructor; much excited for learning this valuable course... I loved the demonstrations of the code executions; Overall, Databricks has been made interesting!
Kashif
October 4, 2022
It is a good course to practice basic data analysis using Scala, PySpark and Spark SQL in databricks environment. Dataset is provide which is good part. Learner can use the dataset to hands on the scala and pyspark
Carlos
March 8, 2022
So basic and opy/paste from cloudera training, furthermore you need an azure databricks and pay as you go when you could to do everything with a sandbox or cloud comunity.
M
December 10, 2021
Good information but a fair amount of verbal repetition and looking around on the screen for things, as if the lecture wasn't practiced before recording.
Akshay
November 20, 2021
This was a good course but only complain here is that Databricks Development and deployment lifecycle for python is missing.
Sahadev
June 23, 2021
Very good course on Databricks...I got to learn a lot of exciting things regarding Databricks and I think it will surely help me in my current and future project...
Shashi
May 22, 2021
Good content. The instructor moves the screens extremely fast and also speaks fast. Please slow down.
SOUVIK
April 25, 2021
very crisp contents , Author does not waste anytime to deep dive into the tutorials , all 3 sections to solve the same exercise with Python , Scala and SQL in Databricks is very informative and useful to learners like me. Great Job and Thanks a lot!
Rubén
April 14, 2021
The information is good, but the explanation is not so well presented, the teacher jumps from screen to screen just saying "do this thing" without further explanation.
ANDRESSA
April 6, 2021
Muito difícil de entender o inglês dele. Fala muito rápido. Precisava falar mais devagar por ter um sotaque tão forte.
Neeraj
November 9, 2020
Instructor seems to be in hurry and as part of every video instructor keeps on marketing about the other courses.
Nedra
March 28, 2020
The instructor has a very heavy accent. It is difficult to understand some of his words. He does seem knowledgeable so far.
Dhileep
March 19, 2020
Good, having what I am expected. But,Many flash content is available, many times search engine blocking the videos. Later this year search engines are not allow flash content.

Coupons

DateDiscountStatus
9/9/2022100% OFF
expired
3/11/2023100% OFF
expired
5/13/2023100% OFF
expired
7/12/2023100% OFF
expired
3/12/2024100% OFF
expired

Charts

Price

Databricks Essentials for Spark Developers (Azure and AWS) - Price chart

Rating

Databricks Essentials for Spark Developers (Azure and AWS) - Ratings chart

Enrollment distribution

Databricks Essentials for Spark Developers (Azure and AWS) - Distribution chart
2511956
udemy ID
8/16/2019
course created date
11/20/2019
course indexed date
Bot
course submited by