Since December 2024, through the joint efforts of the vLLM community and the Ascend team on vLLM, we have completed the Hardware Pluggable RFC. This proposal allows hardware integration into vLLM in a decoupled manner, enabling rapid and modular support for different hardware platforms.

vLLM

The vLLM community and Ascend team have developed a hardware plugin system that enables modular support for different hardware platforms. This approach reduces code complexity, lowers maintenance costs, and enhances extensibility by allowing backend integration in a decoupled manner. The plugin system is validated with projects like vllm-ascend and vllm-spyre. Developers can follow specific steps to integrate new backends using this system.

Introducing vLLM Hardware Plugin, Best Practice from Ascend NPU

How to Integrate a New Backend via vLLM Hardware Plugin Mechanism