Ten Months with Copilot Coding Agent in dotnet/runtime

After ten months using GitHub Copilot Coding Agent (CCA) in the dotnet/runtime repository, the .NET team shares detailed data and lessons from 878 CCA pull requests (535 merged, 67.9% success rate). Key findings include: cleanup and removal tasks have the highest success rate (84.7%), while performance tasks are hardest (54.5%); proper setup instructions dramatically improved success from 41.7% to ~71%; CCA excels at well-scoped mechanical tasks but struggles with architectural judgment; 65.7% of CCA-added lines are test code; and the bottleneck has shifted from code generation to code review. The post covers specific experiments like assigning issues from a phone during a flight, the importance of copilot-instructions.md, and the challenge of AI-generated tests that may encode incorrect behavior.