Kubernetes underpins a large portion of all AI workloads in production. Yet, maintaining GPU nodes and ensuring that applications are running…

NVIDIA DevTalk serves as a vibrant community hub where developers can engage in discussions, seek assistance, and collaborate on projects involving NVIDIA hardware and software. Developers can tap into the collective expertise of the NVIDIA developer community, sharing insights, troubleshooting issues, and exploring best practices for GPU programming and AI development. Additionally, DevTalk provides a platform for developers to showcase their projects, receive feedback, and network with peers, fostering collaboration and knowledge exchange within the NVIDIA ecosystem.

NVIDIA Developer

NVSentinel is an open source monitoring and self-healing system for Kubernetes clusters running GPU workloads. It continuously monitors GPU health using NVIDIA DCGM diagnostics, detects issues like ECC errors and driver crashes, and automatically remediates problems through actions like node quarantine, draining, and triggering external repair workflows. The system uses a modular, pluggable architecture that allows operators to deploy only needed components and integrate with existing remediation pipelines. Currently in experimental phase, it supports all DCGM-compatible NVIDIA data center GPUs including H100, B200, A100, and NVSwitch-based systems.

Automate Kubernetes AI Cluster Health with NVSentinel

A health system for Kubernetes GPU clusters