EvalHub: Because "looks good to me" isn't a benchmark

Enterprise AI teams routinely ship unreliable models because their evaluation processes rely on manual spot checks and fragmented tooling. Five structural problems are identified: tooling fragmentation, unclear measurement goals, reproducibility failures, documentation gaps, and the dev-to-production gap. EvalHub, a Red Hat AI open-source project, addresses all five through a single Kubernetes-deployed orchestration layer that routes evaluation requests to backends like lm-evaluation-harness, Garak, GuideLLM, and MTEB. It introduces versioned evaluation collections for domain-specific benchmarking (e.g., healthcare safety), automatic MLflow experiment tracking, OCI artifact persistence for tamper-evident governance, and a Python SDK with CLI and MCP server support. The same interface works from a local notebook to a production OpenShift cluster via Kueue-based resource management.