AI agents can create convincing ecological models, but you still need to know what you’re doing

A peer-reviewed study tested agentic AI tools (using Roo Code with Claude Sonnet and Kimi K2 models) on three fisheries/ecological modelling tasks in R, running each task 10 times and scoring outputs against a rubric. Key findings: agents can be genuinely useful but require detailed specification sheets, explicit algorithm instructions, and expert oversight. The most dangerous failure mode is professionally formatted output containing subtle methodological errors — such as wrong confidence interval methods or incorrect mortality sequencing — that look correct but are logically flawed. Inexperienced researchers risk producing analyses they cannot evaluate, while even experts may become overconfident. All code and specification sheets are publicly available on GitHub.