kafka 223% Faster (And What We Learned Along the Way)

Platformatic improved their Kafka client for Node.js by 223% after discovering their benchmark methodology was flawed. By fixing measurement issues (per-operation timing, proper delivery tracking, larger sample sizes), they identified real bottlenecks including CRC32C computation, error handling, and metadata request bugs. Key optimizations included switching to native Rust CRC32C implementation, refactoring async error handling, and fixing connection handling. The pure JavaScript implementation now achieves 92,441 op/sec for single messages and 159,828 op/sec for consumption with <2% variance, outperforming native librdkafka bindings by avoiding cross-boundary overhead while maintaining minimal buffer copying and non-blocking event loop usage.