Apache Kafka A-Z with Hands on Learning

Learn Kafka from scratch to advanced through hands on learning

3.70 (45 reviews)
Udemy
platform
English
language
IT Certification
category
387
students
9.5 hours
content
Sep 2021
last update
$59.99
regular price

What you will learn

Learn to handle real-time data feeds using Kafka open-source messaging

Master important points - Topics, Partitions, Brokers, Producers, Consumers

Learn how to build robust streaming applications using Kafka for real-time messaging

Create Producers and Consumers

Write Kafka Streams application

Configure/run Kafka Source and Sink Connectors

Write your own customized Kafka Connector

Configure Standalone and Sink Connector

Build Standalone Application using Kafka and Storm

Create Flume agent for Sending data from Kafka to HDFS

Description

[4-Sep-21 Update] Added code of demos as downloaded resource

Apache Kafka is an open-source distributed stream processing platform that provides high-throughput and low latency real-time messaging. More than 80% of all Fortune 100 companies trust, and use Kafka. Companies like Airbnb, Netflix, Microsoft, Intuit, Target, etc use Kafka extensively.

This course has been aligned with industry best practices and has been created by industry leaders.

This is the an exhaustive course covering A-Z of Kafka:

-Basic concepts and architecture of Kafka

-Kafka Producer and consumer

- Serializer/De-serializer

-Kafka Streams

-Kafka Connect

-Cluster setup and Administrating Kafka

-Kafka Monitoring and Schema registry

-Integration of Kafka with Storm

- Integration of Kafka with Spark and Flume

- Kafka Security

-and Many more concepts in detail


The course contains :

-High quality engaging videos of 9.5 Hrs

-18 Demos

-Quizzes for each lesson

-1 Project


The course will help you design Apache Kafka and learn how Apache Kafka is used to storing and processing multiple nonstop streams of information faster and all the more efficiently.


Learn how to design and introduce Kafka groups, With simple to follow and step by step guidelines.


This Apache Kafka course will help students:

- To learn the all the required knowledge to undertake responsibility for their organization's Kafka group by arranging Kafka producer, consumer, streams, and connectors

- Depicting the design of Kafka & clarify use cases in business

- Start journey in Kafka

Content

Course Introduction

Introduction

Lesson 1: Getting Started with Big Data and Apache Kafka

Learning Objective
Data Evolution Over Time
Big Data Overview
Big Data Analytics
Messaging System
Introduction and Features of Kafka
Kafka Usecases
Kafka Terminologies
Kafka Components
Kafka Architecture
Kafka Clusters
Kafka Industry Usecases
Demo - Install Zookeeper
Demo - Install Kafka
Demo - Setup a Single Node Single Broker Cluster
Demo - Setup a Multi Node Single Broker Cluster
Key Takeaway
Knowledge Check

Lesson 2: Kafka Producer

Learning Objective
Overview of Producer and It's Architecture
Kafka Producer Configuration
Kafka Producer Optional Configuration
Kafka Producer Configuration Objects
Demo - Create a Kafka Producer
Sending Messages
Serializers and Custom Serializers
Demo - Creating a Custom Serializer
Serializers Challenges and Serializing using Apache Avro
Demo - Serializing Using Apache Avro
Partitions
Demo - Setup Custom Partition
Key Takeaway
Knowledge Check

Lesson 3: Kafka Consumer

Learning Objective
Overview of Kafka Consumer
Consumer Groups
Partition Rebalance and Creating a Consumer
Poll loop and Its Functioning
Kafka Configuring Consumer - Part 1
Kafka Consumer Configuration - Part 2
Demo - Create Kafka Consumer
Commit and Offset
Ways of Commiting Offset - Automatic Offset
Ways of Commiting Offset - Commit Current Offset
Ways of Commiting Offset - Asynchronous Commit
Ways of Commiting Offset - Combining Synchronous and Asynchronous Commits
Ways of Commiting Offset - Commit Specified Offset
Rebalance Listeners
Consuming Records with Specific Offset
Deserializers
Demo - Create and Use Custom Deserializer
Key Takeaway
Knowledge Check

Lesson 4: Kafka Operations and Performance Tuning

Learning Objective
Kafka Internals Overview
Replication and Replica Types
Preffered Ladder, Request and Request Processing
Types of Requests
Partition Allocation, File Management and Segments
File Format, Index and Compaction
Kafka Reliability and Reliablity Methods
Broker Configuration for Replication
Producer in Reliable System
Consumer in Reliable System
Key Takeaway
Knowledge Check

Lesson 5: Kafka Cluster Architectures and Administering Kafka

Learning Objective
Cluster Mirroring
Multi Cluster Architecture - Hub, Spokes, and Active Active Architecture
Multi Cluster Architecture - Active-Standby Architecture and Strech Clusters
MirrorMaker Configuration
MirrorMaker Deployment and Tuning
Demo - Setting up MirrorMaker
Administering Kafka - Topic Operations
Administering Kafka - Consumer Group Operations
Dynamic Configuration Changes
Partition Management
Console Producer Tool
Console Consumer Tool
Key Takeaway
Knowledge Check

Lesson 6: Kafka Monitoring and Schema Registry

Learning Objective
Monitoring and Its Importance
Server or Infrastructure Monitoring and Application Monitoring
Kafka Monitoring
Kafka Broker Metrics - Under Replicated Partitions
Kafka Broker Metrics - Others
Topic and Partition Specific Metrics
Logging and Client Monitoring
Producer and Consumer Metrics
Quotas and Lag Monitoring
Monitoring Dashboard
Demo - Setting up Open Source Health Monitor
Kafka Schema Registry
Kafka Component and Architecture
Kafka Schema Registry - Internal working and Use-cases
Kafka Schema Registry Working
Demo - Using Kafka Schema Registry With Kafka
Key Takeaway
Knowledge Check

Lesson 7: Kafka Streams and Kafka Connectors

Learning Objective
Kafka Stream Overview
Kafka Stream
Kafka Stream Architecture and Working
Kafka Stream Components
Kafka Stream Architecture Tasks, Threading Model and Local State Store
Kafka Stream Architecture - Record Buffer
Memory Management and Streaming Data Pipeline
Kafka Stream DSL
K-Stream Operations
K-Table
K-Table Operations
Aggregation and Windowing
Processor Topology and Stream Processor
Stream and Processor APIs
Processor APIs - Create Topology
Kafka Connectors
Standalone and Sink Connector Configuration
Running Kafka Connect
Kafka Connector Distributed Mode
HTTP Rest Interface
Demo - Kafka Connector
Demo - Create an Application using Kafka Streams
Key Takeaway
Knowledge Check

Lesson 8: Integration of Kafka with Storm

Learning Objective
Apache Storm
Real-time Analytics
Apache Storm Architecture
Apache Storm Components
Apache Storm Topology Part - 1
Apache Storm Topology Part - 2
Kafka Spout
Integration of Apache Storm and Kafka
Demo - Simple Standalone Application using Kafka and Storm
Key Takeaway
Knowledge Check

Lesson 9: Kafka Integration with Spark and Flume

Learning Objective
Introduction to Spark
Spark Components
Resilient Distributed Dataset
RDD Operations - Transformation - Map, FlatMap and Filter
RDD Operations - Transformation - Join, Distinct, First and Take
RDD Operations - Actions
Data Sets and Spark Session
Data Sets and Spark Session Operations
Spark Stream
Spark Integration with Kafka
Demo - Running Small Standalone Application in Spark with Kafka
Flume
Flume Connectors
Flume Kafka to HDFS Configuration
Demo - Creating Flume agent Sending data from Kafka to HDFS
Key Takeaway
Knowledge Check

Lesson 10: Admin Client and Securing Kafka

Learning Objective
Admin Client
Demo - Perform Various Admin Tasks using Admin Client
Kafka Security
Kafka Security Component
SASL
Configure SSL in Kafka
Secure using ACLs
Key Takeway
Knowledge Check
Course End Project

Screenshots

Apache Kafka A-Z with Hands on Learning - Screenshot_01Apache Kafka A-Z with Hands on Learning - Screenshot_02Apache Kafka A-Z with Hands on Learning - Screenshot_03Apache Kafka A-Z with Hands on Learning - Screenshot_04

Reviews

Sergio
May 1, 2023
I liked this course, would prefer a real voice instead of robotic and have some more complex examples though

Charts

Price

Apache Kafka A-Z with Hands on Learning - Price chart

Rating

Apache Kafka A-Z with Hands on Learning - Ratings chart

Enrollment distribution

Apache Kafka A-Z with Hands on Learning - Distribution chart

Related Topics

2995138
udemy ID
4/13/2020
course created date
8/2/2020
course indexed date
Bot
course submited by