Amazon SageMaker AI launches optimized generative AI inference recommendations

Amazon SageMaker AI has launched a new inference recommendations capability that automates optimization and benchmarking for generative AI model deployments. Users define their traffic patterns and performance goals (cost, latency, or throughput), and SageMaker analyzes the model architecture, benchmarks configurations across multiple GPU instance types using NVIDIAs AIPerf, and returns deployment-ready configurations with validated metrics including time to first token, inter-token latency, throughput, and cost projections. The feature is available in seven AWS regions.