NVIDIA TensorRT-LLM Enhancements Deliver Massive Large Language Model Speedups on NVIDIA H200

NVIDIA TensorRT-LLM enhancements deliver massive speedups on Llama 2 70B and enable Falcon-180B to run on a single GPU. It achieves a 6.7x performance boost on the H200 GPU for Llama 2 70B and provides excellent inference throughput for Falcon-180B with reduced memory footprint.