The pivot: Testing through new lenses?

Software testing for GenAI requires a paradigm shift from deterministic verification to interpretive, probabilistic approaches. Drawing parallels between museum exhibition curation and GenAI testing, the author proposes 'curatorial testing' as a methodology that embraces multiple valid interpretations, semantic evaluation, and continuous assessment. This approach acknowledges that GenAI outputs are probabilistic and context-dependent, requiring diverse testing teams to explore the probability distribution of outcomes rather than binary pass/fail criteria. Key considerations include understanding training data, prompts, model updates, and other parameters that influence generative outputs, while adapting traditional testing practices to handle the stochastic nature of AI systems.