Apache Kafka for Beginners - Learn Kafka by Hands-On

[MAJOR UPDATE : This course was completely redone on March 2020]

Description:

Learn the fundamentals and advanced concepts of Apache Kafka in this course. This course will give you a good understanding of all the concepts through Hands-On

• This course will walk you through the Kafka concepts step by step

• All the Kafka concepts will be taught through Hands-On

• This course will  increase your technical expertise in Kafka

• This course will give you good exposure to build Kafka Client applications using Java

• This course will be an eyeopener for people who think Kafka is a complex system and you will find yourself in a better place after you complete this course

• This course will transfer you from no Kafka experience to an experienced Kafka user.

Course Overview:

Getting Started with Kafka

• In this section, I will give you all a quick introduction to Apache Kafka, terminologies and different client APIs that are part of Kafka

Download and Install Kafka

• In this section, we will download the Kafka distribution from the Kafka Website.

Understanding Kafka Components and its Internals - (Theory + Hands-On)

In this section, we will explore the Kafka internals from a theoretical perspective followed by hands-on exploring the Kafka internals.

• Learn about the Topics and Partitions in Kafka

• Setup a Local Kafka Cluster with Multiple Brokers

• Producer/Consumer messages in the Kafka Cluster

• Learn about Consumer Offsets and Consumer Groups

• Commit Log and Retention Policy

• Learn about Kafka Load Distribution and Fault Tolerance and Robustness

Kafka Producer API - Hands-On

In this section, we will learn about building the Kafka Producer to producer messages into Kafka using the producer API

• Setting up the base-project for Producer

• Understand how the producer API works

• Produce messages to the Kafka topic using the Synchronous/Asynchronous approach

• Understand how Kafka producer API works behind the scenes

Kafka Producer API - Guaranteed Message Delivery Configurations

In this lecture, we will learn about the different configurations that guarantee message delivery into Kafka.

• Explore some of the producer API Configurations

• Configuring the acks and min.insync.replicas for guaranteed message delivery

• Override the retries and retry-back-off-ms value\

Kafka Consumer API - Hands On

In this section, we will learn about building Kafka consumers to read messages from Kafka using the Consumer API.

• Setting up the base-project for Consumers

• Consuming Messages using Consumer API - poll() loop

• Build Kafka Consumer using the Consumer API

• auto.offset.reset - earliest vs latest

Consumer Groups & Consumer Rebalance

In this section, we will learn about Consumer Groups and Consumer rebalance.

• Scaling the Consumer using Consumer Groups

• Consumer Rebalance

• Consumer Rebalance & max-poll-interval-ms

Consumer Offsets - Default and Manual Offset Management - Hands-On

• Committing Offsets - Default Offset Commit Strategy

• Manually Committing Offsets

• Committing Offsets Synchronously - using commitSync()

• Committing Offsets Asynchronously - using commitAsync()

• Committing Specific Offsets - using CommitSync()

By the end of this, you will have a complete understanding of how Apache Kafka works and how to build Kafka client applications using Producer and Consumer API.