A Comprehensive Study by BentoML on Benchmarking LLM Inference Backends: Performance Analysis of vLLM, LMDeploy, MLC-LLM, TensorRT-LLM, and TGI

A benchmark study by BentoML compares the performance of various inference backends for serving large language models (LLMs) and highlights that LMDeploy consistently delivers superior performance in time to first token (TTFT) and token generation rates.