Insights, experiences, and knowledge sharing on technology, development, and innovation.
Kafka Brokers Discovery, Every Kafka broker is called a bootstrap server. That means that you only need to connect to one broker and you will be connected to the entire cluster. Zookeeper manages brokers. Kafka can't work without Zookeeper
What is AWS CloudFront? CloudFront is a Content Delivery Network (CDN). Origin Access Identity, Edge Location, HTTPS on CloudFront, Lambda@Edge
Kafka stores the offsets at which a consumer group has been reading. The offsets committed live in a Kafka topic named _consumer_offsets. Delivery Semantics for Consumers : Atmost once, Atleast once, Exactly once
What is AWS Placement Groups? Placement Groups are logical groupings or clusters of instances in the selected AWS region.
Kafka Consumers. Consumers read data from a topic. Consumers know which broker to read from. Each consumer within a group reads from exclusive partitions. What if too many Consumers?
What is AWS Autoscaling. Auto Scaling Group. Common problems with Autoscaling and solutions
What is Elastic Load Balancing? Elastic Load Balancing means automatically distributing incoming application traffic across multiple server instances.
Kafka Producers. Producers can choose to receive acknowledgement of data writes.
What is AWS Systems Manager? AWS Systems Manager is a collection of capabilities for configuring and managing your Amazon EC2 instances, on-premises servers and virtual machines, and other AWS resources at scale.
Kafka Brokers and Topics. Concept of Leader for a Partition. At any time only ONE broker can be a leader for a given partition. Example of brokers and partition. Concept of Leader for a partition