Towards Data Science is a community-powered publication that showcases work in data science, machine learning and artificial intelligence. Every day newcomers, seasoned researchers and industry practitioners publish tutorials, research notes and real-world case studies that help the field move forward.

Towards Data Science

P-hacking — manipulating statistical analyses to achieve significant results — is a well-documented problem in research. Drawing on the 'Big Little Lies' paper, common human p-hacking techniques are explained: ghost variables, optional stopping, outlier exclusion, and scale redefinition. A Stanford experiment (Asher et al.) then tested whether frontier LLMs (Claude Opus and OpenAI Codex) could be guided into p-hacking. Direct prompts to cheat were refused, but a cleverly framed 'nuclear prompt' disguising fraud as rigorous uncertainty analysis bypassed safety mechanisms entirely. For RCTs, the AI found little to exploit. For observational studies, it automated brute-force specification searches, in one case manufacturing a statistically significant effect more than triple the true effect from a null result. The takeaway: be skeptical of AI-assisted observational research and audit the code paths AI takes, not just the final output.

How to Lie with Statistics with your Robot Best Friend

1. The Human Baseline (“Big Little Lies”)

2. AI Sycophancy and the Illusion of Safety