Study shows AI integration slowing developers instead of helping

• Findings suggest that developers and organisations should approach AI integration with measured expectations, acknowledging current limitations and the need for continued refinement.

The advent of artificial intelligence (AI) in software engineering workflows has heralded a new era of promise and potential.

Cutting-edge AI coding tools such as Cursor and GitHub Copilot, powered by advanced models from OpenAI, Google DeepMind, Anthropic, and xAI, have rapidly improved their capabilities in writing code, debugging, and testing.

These tools are widely promoted as solutions that can significantly enhance developer productivity by automating routine tasks and accelerating the coding process.

However, a recent study conducted by the non-profit AI research group METR challenges the assumption that AI coding tools universally improve efficiency for experienced software developers.

METR’s randomised controlled trial included sixteen seasoned open source developers who tackled 246 real-world tasks within large codebases they were familiar with.

The study design involved randomly assigning roughly half of the tasks as “AI-allowed,” permitting the use of state-of-the-art AI coding tools such as Cursor Pro.

Unexpected slowdown

The remaining tasks explicitly forbade AI assistance. Prior to the experiment, the developers anticipated that AI integration would reduce task completion time by approximately 24 per cent. Contrary to their expectations, the findings revealed the opposite effect: task completion time increased by 19 per cent when AI was employed.

Several factors may explain this unexpected slowdown. Despite high familiarity with various web-based large language models (94 per cent of participants), only a little over half (56 per cent) had experience with Cursor specifically, the principal AI tool in the study.

Developers were trained in Cursor use before the trial, but time spent learning to effectively prompt AI and waiting for responses appeared to outweigh the time saved by automated code generation.

Moreover, the complexity and scale of the codebases presented additional challenges that current AI tools struggle to navigate efficiently. As a result, instead of streamlining workflows, AI assistance introduced friction and delays.

Need for cautious optimism

Importantly, the researchers refrain from making broad generalisations or dismissing AI’s utility outright. They acknowledge that other large-scale studies have demonstrated productivity gains associated with AI coding tools. Additionally, rapid advancements in AI technology suggest that the present outcomes may evolve in the near future.

METR itself has observed improvements in AI’s ability to handle intricate, long-term coding tasks over recent years. Nevertheless, the study underscores the need for cautious optimism regarding the immediate applicability and effectiveness of AI coding tools.

Beyond productivity concerns, the research highlights other critical considerations. Prior studies have documented that AI-generated code can contain errors and, in some instances, introduce security vulnerabilities.

These risks emphasise that reliance on AI tools must be balanced with rigorous human oversight. Developers should not assume that AI assistance is infallible or that it will unilaterally accelerate their work without careful integration and adaptation.