Apache Kafka for Beginners - Learn Kafka by Hands-On

Deep dive in to Apache Kafka concepts and learn to build Kafka producers/consumers using Java.

4.67 (944 reviews)
Udemy
platform
English
language
Other
category
Apache Kafka for Beginners - Learn Kafka by Hands-On
5,676
students
5 hours
content
Mar 2024
last update
$69.99
regular price

What you will learn

Understand all the Kafka concepts and Kafka core internals.

Learn Kafka internals through practice

Build Kafka Producer applications using Java

Learn the Kafka Producer Techniques for reliable data delivery

Build Kafka Consumer applications using Java

Advance Kafka Consumer Techniques to handle duplicate processing of records

Consumer Rebalance and different techniques to handle them

Consumer techniques to seek to a specific offset

Learn to build Custom Serializer and Deserializer in Kafka

By the end of the course you will have a complete understanding of Apache Kafka and build Kafka consumer applications using Java

Description

Description:

Learn the fundamentals and advanced concepts of Apache Kafka in this course. This course will give you a good understanding of all the concepts through Hands-On

  • This course will walk you through the Kafka concepts step by step

  • All the Kafka concepts will be taught through Hands-On

  • This course will  increase your technical expertise in Kafka

  • This course will give you good exposure to build Kafka Client applications using Java

  • This course will be an eyeopener for people who think Kafka is a complex system and you will find yourself in a better place after you complete this course

  • This course will transfer you from no Kafka experience to an experienced Kafka user.

Course Overview:

Getting Started with Kafka

  • In this section, I will give you all a quick introduction to Apache Kafka, terminologies and different client APIs that are part of Kafka

Download and Install Kafka

  • In this section, we will download the Kafka distribution from the Kafka Website.

Understanding Kafka Components and its Internals - (Theory + Hands-On)

In this section, we will explore the Kafka internals from a theoretical perspective followed by hands-on exploring the Kafka internals.

  • Learn about the Topics and Partitions in Kafka

  • Setup a Local Kafka Cluster with Multiple Brokers

  • Producer/Consumer messages in the Kafka Cluster

  • Learn about Consumer Offsets and Consumer Groups

  • Commit Log and Retention Policy

  • Learn about Kafka Load Distribution and Fault Tolerance and Robustness

Kafka Producer API - Hands-On

In this section, we will learn about building the Kafka Producer to producer messages into Kafka using the producer API

  • Setting up the base-project for Producer

  • Understand how the producer API works

  • Produce messages to the Kafka topic using the Synchronous/Asynchronous approach

  • Understand how Kafka producer API works behind the scenes

Kafka Producer API - Guaranteed Message Delivery Configurations

In this lecture, we will learn about the different configurations that guarantee message delivery into Kafka.

  • Explore some of the producer API Configurations

  • Configuring the acks and min.insync.replicas for guaranteed message delivery

  • Override the retries and retry-back-off-ms value\

Kafka Consumer API - Hands On

In this section, we will learn about building Kafka consumers to read messages from Kafka using the Consumer API.

  • Setting up the base-project for Consumers

  • Consuming Messages using Consumer API - poll() loop

  • Build Kafka Consumer using the Consumer API

  • auto.offset.reset - earliest vs latest

Consumer Groups & Consumer Rebalance

In this section, we will learn about Consumer Groups and Consumer rebalance.

  • Scaling the Consumer using Consumer Groups

  • Consumer Rebalance

  • Consumer Rebalance & max-poll-interval-ms

Consumer Offsets - Default and Manual Offset Management - Hands-On

  • Committing Offsets - Default Offset Commit Strategy

  • Manually Committing Offsets

  • Committing Offsets Synchronously - using commitSync()

  • Committing Offsets Asynchronously - using commitAsync()

  • Committing Specific Offsets - using CommitSync()

By the end of this, you will have a complete understanding of how Apache Kafka works and how to build Kafka client applications using Producer and Consumer API.

Content

Getting Started

About the Course

Getting Started with Kafka

Introduction to Kafka
Kafka Terminologies and Client APIs

Download and Install Kafka

Download and Install Kafka

Understanding Kafka Components and its Internals - (Theory + Hands On)

Kafka Topics and Partitions - Theory
Set Up a ZooKeeper/Kafka Broker in Local
Create Topic, Produce and Consume Messages using the Command Line Interface(CLI)
Produce and Consume Messages with Key
Consumer Offsets
Consumer Groups
Commit Log and Retention Policy
Kafka as a Distributed Streaming System
Setting up a Kafka Cluster in Local with 3 Kafka Brokers
How Kafka Cluster distributes the Client Requests ? - Leader/Follower
How Kafka handles Data Loss ? - Replication and In-Sync-Replica(ISR)
Fault Tolerance and Robustness in Kafka

Kafka Producers

Download and Install Eclipse
Hands on - Kafka Producer coding using Java
Hands on - How to alter the configuration of a topic ?
Hands on - Producer Partitioning Mechanism using Java

Kafka Consumers

Hands on - Kafka Consumer using Java
Hands On - Offset Management in Kafka consumer using Java
Consumer Group in Kafka Consumer - Insight and Benefits
Hands on - Consumer Group using Java
Hands on - Group Id and its Impacts
Hands on - Reset to a specific Offset value in Kafka Consumer using Java
Kafka Consumer - Additional Configurations

Kafka Client - GUI Tool

Kafka - GUI Client

Apache Camel + Kafka Integration

Camel - Quick Introduction
Hands on - Simple Camel Example
Camel Architecture
Hands on - Kafka Consumer using Apache Camel
Hands on - Kafka Producer using Apache Camel

Apache Spark + Kafka Integration

Spark - Spark Streaming - Quick Introduction
Hands on - Kafka Consumer using Spark Streaming

Apache Kafka - Additional Configurations

Hands on - How to delete a topic ?

Apache Kafka + Spring Boot Integration

About this Section
Spring - Quick Introduction
Spring Boot - Quick Introduction
Simple Spring Boot App - Hands on Part 1
Simple Spring Boot App - Hands on Part 2
Spring Boot - Kafka Consumer - Hands On
Spring Boot - Kafka Producer Hands On
Spring Boot - Kafka Consumer Manual Offset Commit - Hands on
Spring Boot - Kafka Consumer Group - Hands on

Docker - Dockerize Kafka, Zookeeper, Producer and Consumer

Docker - Quick Introduction
How to Install Docker Tool Box on Mac ?
How to Install Docker Tool Box on Windows 10?
Hands on - How to create a docker image for Kafka Producer and Consumer?
How to create an account in Docker Hub and push the images to Docker Hub ?
Hands on - Running basic docker commands
Hands on - How to run a zookeeper/Kafka in Docker ?
Demo - Connect to Dockerized kafka/zookeeper using Dockerized Spring boot app

Apache Kafka - Security and Authentication

Why Kafka security ?
Hands on - How to enable SSL in Kafka Broker?
Hands on - How to access the SSL secured topic using Console Producer/Consumer ?
Hands on - How to configure SSL in Kafka Producer using Java?
Hands on - How to configure SSL in Kafka Consumer using Java?

Bonus Section

Coupon for my Other Courses

Conclusion

Congratulations !

Reviews

Irish
March 28, 2023
This is my 2nd course with Dilip and I am loving it. The lessons are easy to follow and clear. The Q&A also is active when I encounter some issues, he responds immediately unlike others. The most important part is it's hands-on where we can learn more while listening and doing it on our desktop. Excited to start your other courses too. :) Thanks so much for this wonderful course.
Kankshi
February 1, 2022
Content and presentation is not relevant, I don't understand, what you are explaining, Kafka or Microservices.
Kaustubh
December 23, 2021
Lots of errors are coming while doing for windows user. i have confirmed this with one of my friend who has taken the same course. he left in between. this course is not at all help full. trainer was running like some one is behind him. i have to make video slow and watch. waste of time and money.
TANVIR
October 14, 2021
Best course on Kafka in Udemy! The instructor is so knowledgeable and doesn't just state the facts but also shows practically. I am so impressed!!!!
Anish
September 19, 2021
Hey Dilip, This is actually the course that I was looking for. Personally, I loved this course. You explained all the Kafka core that is needed for Java Developers. All the things were clean and clear about Kafka. Thanks for making this course. :)
Mehrdad
March 14, 2021
A really great and professional representation. Specially the slides and figures has helped to achieve a good understanding of Kafka. Excellent. Thank you very much.
Jovan
February 3, 2021
This is a very good introduction to Kafka Producer and Consumer APIs. Highly recommended for programmers getting started with Kafka. I would say that it would be good to get some intuition about Kafka before taking this course by using various commands and console producer/consumer. I'm looking forward to taking Dilip's other courses.
Nitish
November 4, 2017
Course started with good solid points about kafka. I am waiting for other chapters to complete. I really like the content.
Tobias
October 19, 2017
The course is a overall very good experience. The instructor presents the topic very clearly. However, at the Spring section he is jumping a bit to fast. He covers also some more technologies than Kafka, which is integrated very nicely and is very useful indeed. I highly recommend this course if you want to learn Kafka. However, I'm not able to estimate how deep he is diving into the topic. As often, I'm guessing that one has to check further documentation or courses.
Ashu
August 25, 2017
Kafka Architecture Internals should be explain in more details. In this tutorial the Author has just told about Kafka Cluster as part of Kafka Architecture.
Maneesh
July 15, 2017
Completed 2 sections only. But so FAR is 5 Star course. Short and to the point. Let's see how the journey goes. Spring boot kafka setup section is good , but i expected some more explanation about the consumer/producer setup. The explanations looked shallow to me. hope he adds more description to it. Updated version of this course is more crisp and clear with lot more information .
Mahendra
July 13, 2017
Good coverage of Kafka topic but needs more screen displays and detail steps for installation and various installation scenarios (eg Kafka cluster vs single node)
aaron
June 26, 2017
Useful, not just for beginners, I had to run the video at 1.25 to 1.5 speed though as the original speech was slow
Gaurav
June 24, 2017
architecture should be explained more deeply, somebody who have knowledge about kafka, he/she expects more deep understanding
Nikhil
June 15, 2017
Am glad that I have enrolled in this course. Really liked Dilip's delivery and knowledgable instructor with clear objective.Keep up the good work !

Charts

Price

Apache Kafka for Beginners - Learn Kafka by Hands-On - Price chart

Rating

Apache Kafka for Beginners - Learn Kafka by Hands-On - Ratings chart

Enrollment distribution

Apache Kafka for Beginners - Learn Kafka by Hands-On - Distribution chart
1146104
udemy ID
3/15/2017
course created date
11/22/2019
course indexed date
Bot
course submited by