Experiment: Generating “Random” Test Data

An experiment testing Google's Gemini 2.5 ability to generate random test data reveals significant biases and limitations. When asked to create 30 random postal codes from randomly selected countries, the AI produced heavily biased results favoring English-speaking and European countries, used well-known postal codes like 90210, and showed patterns inconsistent with true randomness. The experiment demonstrates several AI syndromes including incongruence, non-responsiveness, and unteachability, highlighting the importance of carefully validating AI-generated test data and considering alternative approaches like having AI generate code to produce data rather than generating data directly.