The Kubernetes community has launched the Certified Kubernetes AI Conformance program, a formal extension of standard Kubernetes conformance aimed at making AI and ML workloads actually portable across cloud providers.
Right now, running a distributed training job or inference service on one cluster doesn’t mean it’ll behave the same way on another. GPU drivers differ, network setups vary, autoscaling behaviors diverge. The new program tries to fix that by defining a common baseline that certified platforms must meet.
<h2>What the program actually covers</h2>
The initial requirements focus on four areas:
Dynamic Resource Allocation (DRA) gives workloads fine-grained control over GPUs and TPUs, replacing the older, blunter device plugin model. This is the foundation everything else builds on.
All-or-nothing scheduling via Kueue addresses a real pain point in distributed training: jobs that partially schedule and then deadlock waiting for the remaining resources. Kueue holds jobs in a queue until all required resources are available before letting anything start.
Custom-metric autoscaling handles inference workloads, which don’t scale well on CPU or memory metrics alone. The program standardizes how clusters expose and act on inference-specific signals.
Standardized accelerator observability means GPU and TPU metrics are exposed consistently, so monitoring and alerting tools don’t need to be reconfigured for each provider.
<h2>Why inference is the priority</h2>
Two-thirds of compute is expected to be dedicated to inference by end of 2026. Training gets most of the attention, but inference is where most production workloads actually live, and it has different scaling and resource characteristics. The conformance program reflects that shift.
The newly launched llm-d project, now in CNCF incubation, is directly relevant here. It integrates vLLM into Kubernetes clusters and is expected to collaborate with the conformance program as inference requirements develop.
<h2>Who’s involved and what’s next</h2>
The program is led by contributors from Google, Microsoft, Red Hat, and Kubermatic, developed in the open. GKE and AKS are already adopting it. Red Hat and Nvidia are among the early participants.
Networking and storage requirements are planned as the program matures. Automated certification testing and expanded inference and security standards are on the roadmap for later in 2026.
The honest caveat: conformance programs are only as useful as their adoption. The fact that major cloud providers are moving early is a good sign, but the real test is whether the certification actually constrains behavior or just becomes a checkbox. That’ll take time to see.

Collections

The Kubernetes community has launched the Certified Kubernetes AI Conformance program to standardize how AI and ML workloads run across cloud providers. The program covers four key areas: Dynamic Resource Allocation (DRA) for fine-grained GPU/TPU control, all-or-nothing scheduling via Kueue to prevent distributed training deadlocks, custom-metric autoscaling for inference workloads, and standardized accelerator observability. Inference is the primary focus given two-thirds of compute is expected to be dedicated to it by 2026. GKE and AKS are already adopting the program, with Google, Microsoft, Red Hat, and Nvidia among early participants. Networking and storage requirements, automated certification testing, and expanded inference standards are planned for later in 2026.

Kubernetes launches AI conformance program to standardize GPU workloads across cloud providers