This time, I discussed how to setup and scale message processing when you have multi-tenant environment or other dynamic setup. You'll see considerations where to scale horizontally, where vertically and why that matters to you

Architecture Weekly's publication is a central hub for architecture enthusiasts, offering insights into architectural design, urban planning, and sustainable architecture practices. Through articles, case studies, and architectural critiques, Architecture Weekly offers insights into iconic buildings, architectural trends, and innovative design solutions. Readers can learn about architectural history, design principles, and environmental sustainability initiatives to gain a deeper understanding of the built environment and its impact on society. Additionally, Architecture Weekly provides updates on the latest architectural projects, design competitions, and industry events to help architects and designers stay informed and inspired.

Architecture Weekly

Multi-tenant message processing requires careful workload distribution strategies. Two main approaches exist: horizontal scaling through physical sharding (better isolation, higher cost) and vertical scaling through partitioning within containers (cheaper, easier management, potential noisy neighbor issues). Dynamic distribution can use consistent hashing to route messages based on tenant ID or other properties. While sophisticated distributed consensus algorithms like Raft exist, leveraging mature messaging systems (Kafka, RabbitMQ) for complex distribution is often more practical than building custom solutions. Partitioned processors with worker threads and distributed locking provide sufficient flexibility for most use cases.

Multi-tenancy and dynamic messaging workload distribution

Distributing the workload dynamically between processors

What options do we have for such a multi-tenant setup?