What you'll learn:
- Understand all the Kafka concepts and Kafka core internals.
- Learn Kafka internals through practice
- Build Kafka Producer applications using Java
- Learn the Kafka Producer Techniques for reliable data delivery
- Build Kafka Consumer applications using Java
- Advance Kafka Consumer Techniques to handle duplicate processing of records
- Consumer Rebalance and different techniques to handle them
- Consumer techniques to seek to a specific offset
- Learn to build Custom Serializer and Deserializer in Kafka
- By the end of the course you will have a complete understanding of Apache Kafka and build Kafka consumer applications using Java
Description:
Learn the fundamentalsand advancedconcepts of Apache Kafka in this course. This course will give you a good understanding of all the concepts through Hands-On
This course will walk you through the Kafka concepts step by step
All the Kafka concepts will be taught through Hands-On
This course will increase your technical expertise in Kafka
This course will give you good exposure to build Kafka Client applications using Java
This course will be an eyeopener for people who think Kafka is a complex system and you will find yourself in a better place after you complete this course
This course will transfer you from no Kafka experience to an experienced Kafka user.
Course Overview:
Getting Started with Kafka
In this section, I will give you all a quick introduction to Apache Kafka, terminologies and different client APIs that are part of Kafka
Download and Install Kafka
In this section, we will download the Kafka distribution from the Kafka Website.
Understanding Kafka Components and its Internals - (Theory + Hands-On)
In this section, we will explore the Kafka internals from a theoretical perspective followed by hands-on exploring the Kafka internals.
Learn about the Topics and Partitions in Kafka
Setup a Local Kafka Cluster with Multiple Brokers
Producer/Consumer messages in the Kafka Cluster
Learn about Consumer Offsets and Consumer Groups
Commit Log and Retention Policy
Learn about Kafka Load Distribution and Fault Tolerance and Robustness
Kafka Producer API - Hands-On
In this section, we will learn about building the Kafka Producer to producer messages into Kafka using the producer API
Setting up the base-project for Producer
Understand how the producer API works
Produce messages to the Kafka topic using the Synchronous/Asynchronous approach
Understand how Kafka producer API works behind the scenes
Kafka Producer API - Guaranteed Message Delivery Configurations
In this lecture, we will learn about the different configurations that guarantee message delivery into Kafka.
Explore some of the producer API Configurations
Configuring the acks and min.insync.replicas for guaranteed message delivery
Override the retries and retry-back-off-ms value\
Kafka Consumer API - Hands On
In this section, we will learn about building Kafka consumers to read messages from Kafka using the Consumer API.
Setting up the base-project for Consumers
Consuming Messages using Consumer API - poll() loop
Build Kafka Consumer using the Consumer API
auto.offset.reset - earliest vs latest
Consumer Groups & Consumer Rebalance
In this section, we will learn about Consumer Groups and Consumer rebalance.
Scaling the Consumer using Consumer Groups
Consumer Rebalance
Consumer Rebalance & max-poll-interval-ms
Consumer Offsets - Default and Manual Offset Management - Hands-On
Committing Offsets - Default Offset Commit Strategy
Manually Committing Offsets
Committing Offsets Synchronously - using commitSync()
Committing Offsets Asynchronously - using commitAsync()
Committing Specific Offsets - using CommitSync()
By the end of this, you will have a complete understanding of how Apache Kafka works and how to build Kafka client applications using Producer and Consumer API.
