Multi-tenant message processing requires careful workload distribution strategies. Two main approaches exist: horizontal scaling through physical sharding (better isolation, higher cost) and vertical scaling through partitioning within containers (cheaper, easier management, potential noisy neighbor issues). Dynamic distribution can use consistent hashing to route messages based on tenant ID or other properties. While sophisticated distributed consensus algorithms like Raft exist, leveraging mature messaging systems (Kafka, RabbitMQ) for complex distribution is often more practical than building custom solutions. Partitioned processors with worker threads and distributed locking provide sufficient flexibility for most use cases.
Table of contents
Distributing the workload dynamically between processorsWhat options do we have for such a multi-tenant setup?Sort: