Colocating Input Partitions with Kafka Streams When Consuming Multiple Topics: Sub-Topology Matters!

Explores a production issue where Kafka Streams failed to colocate matching partitions from two topics on the same instance, breaking cache locality. The root cause was implicit creation of separate sub-topologies for each topic. The solution involved replacing a local Guava cache with a shared Kafka Streams state store, which unified the sub-topologies and enforced partition colocation. This architectural change ensured identical keys from both topics were processed by the same instance, significantly reducing redundant external API calls and improving performance.