Apache Kafka Series - Learn Apache Kafka for Beginners v2

START HERE: Learn Apache Kafka 2.0 Ecosystem, Core Concepts, Real World Java Producers/Consumers & Big Data Architecture

4.69 (20959 reviews)



7.5 hours


Feb 2021

Last Update
Regular Price

What you will learn

Understand Apache Kafka Ecosystem, Architecture, Core Concepts and Operations

Master Concepts such as Topics, Partitions, Brokers, Producers, Consumers

Start a personal Kafka development environment

Learn major CLIs: kafka-topics, kafka-console-producer, kafka-console-consumer, kafka-consumer-groups, kafka-configs

Create your Producers and Consumers in Java to interact with Kafka

Program a Real World Twitter Producer & ElasticSearch Consumer

Extended APIs Overview (Kafka Connect, Kafka Streams), Case Studies and Big Data Architecture

Practice and Understand Log Compaction


UPDATE SEPTEMBER 2018: Course newly recorded with Kafka 2.0!

Welcome to the Apache Kafka Series! Join a community of 20,000+ students learning Kafka.

Apache Kafka has become the leading distributed data streaming enterprise  big data technology. Kafka is used in production by over 33% of the Fortune 500 companies such as Netflix, Airbnb, Uber, Walmart and LinkedIn.

To learn Kafka easily, step-by-step, you have come to the right place! No prior Kafka knowledge is required


If you look at the documentation, you can see that Apache Kafka is not easy to learn...

Thanks to my several years of experience in Kafka and Big Data, I wanted to make learning Kafka accessible to everyone.

We'll take a step-by-step approach to learn all the fundamentals of Apache Kafka.
At the end of this course, you'll be productive and you'll know the following:

  • The Apache Kafka Ecosystem Architecture

  • The Kafka Core Concepts: Topics, Partitions, Brokers, Replicas, Producers, Consumers, and more!

  • Launch your own Kafka cluster in no time using native Kafka binaries – Windows / MacOS X / Linux 

  • Learn and Practice using the Kafka Command Line Interface (CLI)

  • Code Producer and Consumers using the Java API 

  • Real world project using Twitter as a source of data for a producer and ElasticSearch as a sink for our consumer

  • Overview of Advanced APIs (Kafka Connect, Kafka Streams)

  • Real World Case Studies and Big  Use cases

  • Overview of Advanced Kafka for Administrators

  • Advanced Topic Configurations

  • Annexes (starting a Kafka cluster locally, using Docker, etc...)

Note: The hands-on section is based on Java, which is the native Kafka programming language. But, good news! Your learning in Java will be completely applicable to other programming languages, such as Python, C#, Node.js or Scala, and Big Data frameworks such as Spark, NiFi or Akka


The Kafka for Beginners course is the best place to start your Kafka learning journey... Ask other students!

  • 5 stars: "That's one of the most high-quality on-line courses I ever took" – Paul L.

  • 5 stars: "This training was awesome and learned many things about KAFKA though having 0 years of experience in this technology" – Puneet G.

  • 5 stars: "Outstanding on all fronts! I wish all courses were that well presented. I especially appreciate the hands-on sections. I was able to get everything up and running with ease. Highly recommend this course and this instructor!" – David G.   

You can take this course risk-free and if you don't like it, you can get a refund anytime in the first 30 days!



Stephane Maarek is the instructor of this course. He is a Kafka Expert, guest author on the Confluent Blog and Speaker at the Kafka Summit SF 2018. He's the author of the highly-rated Apache Kafka Series on Udemy, having taught already to 40,000+ students and received 12,000+ reviews.

He shares all his Kafka knowledge on the platform, taking the time to explain every concept and provide students with both theoretical and practical dimensions. You are in good hands!


This Course Also Comes With:

  • Lifetime Access to All Future Updates

  • A responsive instructor in the Q&A Section

  • Links to interesting articles, and lots of good code to base your next applications onto

  • Udemy Certificate of Completion Ready for Download   

This is the course that could improve your career!  

Apache Kafka is a skill in high demand and there are not enough people to fulfill all the open positions. You can boost your income, take on new roles and fun challenges. Many of my students are now the Kafka experts of their companies!

I hope to see you inside the course!


Note: Looking for more advanced Kafka concepts? There are many volumes in the Apache Kafka Series:

  • Learn Kafka for Beginners v2 (this course - great to start)

  • Kafka Connect Hands On Learning

  • Kafka Streams for Data Processing

  • Kafka Cluster Setup & Administration

  • Confluent Schema Registry & Kafka REST Proxy

  • Kafka Security (SSL SASL ACL)

  • Kafka Monitoring and Operations

Happy learning!


Apache Kafka Series - Learn Apache Kafka for Beginners v2
Apache Kafka Series - Learn Apache Kafka for Beginners v2
Apache Kafka Series - Learn Apache Kafka for Beginners v2
Apache Kafka Series - Learn Apache Kafka for Beginners v2


Kafka Introduction


Apache Kafka in 5 minutes

Course Objectives

Important Message

Code Download

Code Download

====== Kafka Fundamentals ======

Kafka Fundamentals

Kafka Theory

Kafka Theory Overview

Topics, Partitions and Offsets

Brokers and Topics

Topic Replication

Producers and Message Keys

Consumers & Consumer Groups

Consumer Offsets & Delivery Semantics

Kafka Broker Discovery


Kafka Guarantees

Theory Roundup

Quiz on Theory

Starting Kafka

Important: Starting Kafka

FAQ for Setup Problems

Mac OS X - Download and Setup Kafka in PATH

Mac OS X - Using brew

Mac OS X - Start Zookeeper and Kafka

Mac OS X - Summary

Linux - Download and Setup Kafka in PATH

Linux - Start Zookeeper and Kafka

Linux - Summary

Windows - Download Kafka and PATH Setup

Windows - Start Zookeeper & Kafka

Windows - Summary

CLI (Command Line Interface) 101

CLI Introduction


Kafka Topics CLI

Kafka Console Producer CLI

Kafka Console Consumer CLI

Kafka Consumers in Group

Kafka Consumer Groups CLI

Resetting Offsets

CLI Options that are good to know

What about UIs? Conduktor

KafkaCat as a replacement for Kafka CLI

Quiz on CLI

Kafka Java Programming 101

Intro to Kafka Programming

Installing Java 8 & IntelliJ Community Edition

Creating Kafka Project

Java Producer

Java Producer Callbacks

Java Producer with Keys

Java Consumer

Java Consumer inside Consumer Group

Java Consumer with Threads

Java Consumer Seek and Assign

Client Bi-Directional Compatibility

Configuring Producers and Consumers

Quiz on Java Programming 101

===== Kafka Real World Project =====

Real World Project Overview

Real World Exercise

Kafka Twitter Producer & Advanced Configurations

Producer and Advanced Configurations Overview

Twitter Setup

Producer Part 1 - Writing Twitter Client

Producer Part 2 - Writing the Kafka Producer

Producer Configurations Introduction

acks & min.insync.replicas

retries, delivery.timeout.ms & max.in.flight.requests.per.connection

Idempotent Producer

Producer Part 3 - Safe Producer

Producer Compression

Producer Batching

Producer Part 4 - High Throughput Producer

Producer Default Partitions and Key Hashing

[Advanced] max.block.ms and buffer.memory

Quiz on Producer Configurations

Refactoring the Project

Kafka ElasticSearch Consumer & Advanced Configurations

Consumer and Advanced Configuration Overview

Setting up ElasticSearch in the Cloud

ElasticSearch 101

Consumer Part 1 - Setup Project

Consumer Part 2 - Write the Consumer & Send to ElasticSearch

Delivery Semantics for Consumers

Consumer Part 3 - Idempotence

Consumer Poll Behaviour

Consumer Offset Commit Strategies

Consumer Part 4 - Manual Commit of Offsets

Consumer Part 5 - Performance Improvement using Batching

Consumer Offsets Reset Behaviour

Consumer Part 6 - Replaying Data

Consumer Internal Threads

Quiz on Consumer Configurations

===== Kafka Ecosystem & Real-World Architectures =====

Kafka in the Real World

Kafka Extended APIs for Developers

Kafka Connect Introduction

Kafka Connect Twitter Hands-On

Kafka Streams Introduction

Kafka Streams Hands-On

Kafka Schema Registry Introduction

Which Kafka API should I use?

Quiz on Kafka Extended APIs

Real World Insights and Case Studies (Big Data / Fast Data)

Choosing Partition Count & Replication Factor

Kafka Topics Naming Convention

Case Study - MovieFlix

Case Study - GetTaxi

Case Study - MySocialMedia

Case Study - MyBank

Case Study - Big Data Ingestion

Case Study - Logging and Metrics Aggregation

Kafka in the Enterprise for Admins

Kafka Cluster Setup High Level Architecture Overview

Kafka Monitoring & Operations

Kafka Security

Kafka Multi Cluster & MirrorMaker

===== Advanced Kafka =====

Advanced Kafka

Advanced Topics Configurations

Changing a Topic Configuration

Segment and Indexes

Log Cleanup Policies

Log Cleanup Delete

Log Compaction Theory

Log Compaction Practice

min.insync.replicas reminder

Unclean Leader Election

===== Annexes =====

What are annexes?

Annex - Starting Kafka Differently

Annex 1 - Overview

Starting Kafka with the Confluent CLI

Starting a multi broker Kafka Cluster using Binaries

Note on Docker Setup

Start Kafka Development environment using Docker

Starting a multi broker Kafka Cluster using Docker

Kafka Advertised Host Setting

Starting Kafka on a Remote Machine

===== Next Steps =====

What's Next?


Bonus Lecture: Student Special Coupons for my Other courses


Panagiotis9 October 2020

Great instructor, have been following many of his courses and each one is better than the last. Clear and practical examples provided make his courses pretty easy to follow and understand.

Raymond7 October 2020

I've done other courses about Kafka on other platforms and I think this is an awesome course, it can be a little bit advance in the lastest lectures but that it is ok because those are topics that we are going the search in the future.

Chris7 October 2020

Great course, gives a good introduction of all the topics around Kafka! I am looking forward for the deep dive courses!

Deepanshu6 October 2020

It was really a very good learning, just thought of having troubleshooting lessons more. Few live issues where we can know how to fix the issues.

Viacheslav5 October 2020

The curse is very interesting and helpful. There is only one issue in the curse. It's not possible to obtain Twitter development account but any way for me it was enough to watch video and investigate code without running.

Ningareddy27 February 2020

Excellent course to learn core concepts of kafka and get some insight about advanced concepts like connect, stream, schema registry etc. The explanation is to the point and you made sure that learner is focused on the topic without diverting. Hands-on examples along with theory explanation is superb!. Overall it is excellent course to learn kafka basics. I loved it. Thank you for this excellent course. Love to lean a lot from you.

Lalita26 February 2020

I was looking for a tutorial of less duration and that explains to me Kafka to get my skills to a level to start practicing it and this course helped me fulfill it.

Olga23 February 2020

This course is exactly what I was looking for: it's concise and contains only the "juice". It's great to get familiar with Kafka and get ready.

Konrad22 February 2020

I really learned a lot and specially every step was explained in detail to reproduce it on an own machine.

Alberto21 February 2020

La teoria ha sido explicada muy bien y ademas los datos adicionales sobre el borrado de las particiones (la numeracion) que no vuelva a reiniciarse en cero es de considerar. Gracias.

Vipul20 February 2020

Stephen has covered a lot of topics in this course. Even though I have been working for almost a year with Kafka, I still got to learn a lot.

Kiran20 February 2020

Really loved the case studies , stephen has created a very detailed course in kafka for beginners . Anyone who wants to learn about kafka should definitely try this course , highly recommend it . Thanks Stephen !

Binson20 February 2020

Like Kafka, This course is powerful ,clear. Author is excellent ,crystal clear explanation . simple language . So highly recommending .

Dkovalbuzz19 February 2020

Awesome course for beginners! This course is good if you wish to get your hands dirty with Kafka, as well as get a solid understanding of Kafka fundamentals and the theory behind.

Mala18 February 2020

It was an amazing course and really helped me understand the concepts and get the hands on with so much of clarity.


Expired2/8/202091% OFF


Udemy ID


Course created date


Course Indexed date
Course Submitted by

Android PlayStore
Apple Appstore