I'm Scott Kirkpatrick

Kafka is fast. It’s also pretty lean in production too, taking up very little heap space and CPU. As an administrator of thousands of Kafka clusters, it’s gotten to a point where the behavior of the brokers are fairly known and there’s not really any major changes in resource usage patterns over time, even with new applications leveraging Kafka for their data. But there’s always outliers. One new feature relying on Kafka managed to bring a canary cluster to its knees, taking out the entire set of brokers deployed there. Read more