TechGry
kafka broker-discovery zookeeper bootstrap-server metadata

Kafka Broker Discovery and Zookeeper

Kafka Brokers Discovery, Every Kafka broker is called a bootstrap server. That means that you only need to connect to one broker and you will be connected to the entire cluster. Zookeeper manages brokers. Kafka can't work without Zookeeper

Vishnu Sunkari
California USA
1 min read

Kafka Broker Discovery

  • Every Kafka broker is called a bootstrap server.
  • That means that you only need to connect to one broker and you will be connected to the entire cluster.
  • Each broker knows about all brokers, topics and partitions (metadata)

kafka broker_discovery

Zookeeper

  • Zookeeper manages brokers (keeps a list of them)
  • Zookeeper helps in performing leader election for partitions
  • Zookeeper sends notifications to Kafka in case of changes (eg new topic, broker dies, broker comes up, delete topics etc)
  • Kakfa can’t work without Zookeeper
  • Zookeeper by design operates with an odd number of servers(3, 5, 7)
  • Zookeeper has a leader (handle writes) the rest of the servers are followers (handle reads)
  • Zookeeper does NOT store consumer offsets with Kafka > v0.10)

kafka zookeeper

Comments

Link copied to clipboard!