The earlier versions of vLLM Semantic Router relied on classification-based routing, a straightforward approach where user queries are classified into one of 14 MMLU domain categories, and then routed to corresponding models. While this worked for basic scenarios, we quickly discovered its limitations when building production AI systems for enterprises.

vLLM

vLLM introduces Signal-Decision Architecture, a new approach to semantic routing that replaces fixed classification-based routing with multi-dimensional signal extraction. The architecture combines keyword, embedding, and domain signals with flexible AND/OR logic to enable unlimited routing decisions. It includes built-in plugins for caching, security, and compliance, and uses Kubernetes CRDs for cloud-native deployment. This enables enterprises to scale from 14 fixed categories to hundreds of specialized routing rules with priority-based selection and plugin orchestration.

Signal-Decision Driven Architecture: Reshaping Semantic Routing at Scale

The Problem: Why Classification-Based Routing Doesn’t Scale