Kluge - The Mysteries of Kafka
a bit of IT lore to help me remember it all: Kafka: Google says: Kafka is used to build real-time streaming data pipelines and real-time streaming applications. A data pipeline reliably processes and moves data from one system to another, and a streaming application is an application that consumes streams of data. The Genesis Project: Design The "Design" references two solid reads: Big Data, and regarding the Kafka Design, disk (OS pagecache) potentially as fast as in-memory RAM, Disk Reads, and Multi-CPUs Code - the all-important Producer, careful, this gets deep: KafkaProducer