Study Finds AI Coding Agents Increase Code Output but Not Overall Software Productivity

Image: Ars Technica · Source

Research reveals AI coding assistants generate more code but trigger longer human reviews, limiting gains in software delivery and employment changes.

Modern AI coding assistants and agents have proven adept at generating large volumes of functional code quickly, yet a recent study indicates that this increased coding output does not directly translate into higher overall software production. Harvard University researchers Fiona Chen and James Stratton analyzed extensive data from over 700 software development firms, encompassing more than 700,000 employees, to understand how AI coding tools impact software development workflows between 2021 and early 2026.

Using data from Jellyfish, a platform that tracks detailed engineering team activities, the researchers observed that after firms introduced AI coding agents—autonomous systems that write and submit code based on prompts—there was a notable 30 percent increase in lines of code produced, alongside rises of 20 percent in commits and 23 percent in pull requests. Despite this surge, measures of software output such as issue and feature resolution rates remained statistically unchanged.

The study attributes this discrepancy to a significant bottleneck in the human code review stage. After AI adoption, the time taken from pull request submission to merge extended by 49 percent. Moreover, requests for changes nearly doubled, and comments per pull request increased by 35 percent, indicating more intensive scrutiny of AI-generated code. This increased review effort led to a 14 percent rise in personnel involved in code reviews, without observable changes in overall employment levels.

While AI tools have also been adopted to assist in code review, their impact remains limited; by March 2026, AI agents contributed to roughly a quarter of review comments and an even smaller fraction of pull requests. Human reviewers continue to bear the bulk of validation responsibilities.

These findings highlight a current trade-off: AI coding agents speed up code creation but demand substantially more human review time, negating expected productivity gains. Given that AI coding adoption is relatively recent and evolving, further improvements in integration strategies could optimize this balance. However, as it stands, companies must consider whether the investment in AI coding tools justifies the additional review workload they impose.