CNCF Expands Efforts to Run AI Inference Workloads on Kubernetes Clusters

The CNCF and Red Hat announced at KubeCon + CloudNativeCon Europe 2026 that Red Hat's llm-d framework for distributed AI inference has been contributed to the CNCF. The foundation also published stricter Kubernetes AI Requirements (KARs) v1.35, now mandating stable in-place pod resizing and workload-aware scheduling. New technical benchmarks include high-performance pod-to-pod communication, advanced inference ingress, and disaggregated inference support. A 'Verify Conformance Bot' for third-party validation is planned, along with future Sovereign AI standards. The KAR program now counts 31 certified platforms. Additional ecosystem projects include AWS's Kube Resource Orchestrator (KRO), the Kueue job queueing system, and NVIDIA's open source AI Cluster Runtime (AICR). CNCF executive director Jonathan Bryce emphasized Kubernetes as the future dominant platform for AI inference, including edge deployments with smaller LLMs.