Introduction to Apache Kafka for Event Streaming > 자유게시판

Apache Kafka is a distributed event streaming platform. Topics organize related events with configurable partitions. Producers publish events to topics with optional keys. Consumers subscribe to topics and process events. Consumer groups balance processing across instances. Partitions enable parallel processing and ordering guarantees. Offsets track consumer position in partition. Brokers form Kafka cluster for fault tolerance. Replication factor ensures data durability. ZooKeeper manages cluster metadata and leader election. ZooKeeper in newer versions. Message retention policies manage storage by time or size. Compacted topics retain latest value per key. Exactly-once semantics prevent duplicate processing. Schema Registry manages Avro, JSON, and Protobuf schemas. Kafka Connect integrates with external systems. Kafka Streams processes streams within applications. ksqlDB enables SQL-based stream processing. MirrorMaker replicates data across clusters. Monitoring tools include Kafka Manager and Confluent Control Center. Kafka is the backbone for event-driven architectures and data pipelines.