It’s like a brand new telling of the “Tortoise and the Hare”: A gaggle of skilled software program engineers entered into an experiment the place they had been tasked with finishing a few of their work with the assistance of AI instruments. Considering just like the speedy hare, the builders anticipated AI to expedite their work and enhance productiveness. As a substitute, the expertise slowed them down extra. The AI-free tortoise method, within the context of the experiment, would have been sooner.
The outcomes of this experiment, a part of a current examine, got here as a shock to the software program builders tasked with utilizing AI—and to the examine’s authors, Joel Becker and Nate Rush, technical employees members of nonprofit expertise analysis group Mannequin Analysis and Menace Analysis (METR).
The researchers enlisted 16 software program builders, who had a mean of 5 years of expertise, to conduct 246 duties, every one part of initiatives on which they had been already working. For half the duties, the builders had been allowed to make use of AI instruments—most of them chosen code editor Cursor Professional or Claude 3.5/3.7 Sonnet—and for the opposite half, the builders carried out the duties on their very own.
Believing the AI instruments would make them extra productive, the software program builders predicted the expertise would cut back their job completion time by a mean of 24%. As a substitute, AI resulted of their job time ballooning to 19% higher than once they weren’t utilizing the expertise.
“While I like to believe that my productivity didn’t suffer while using AI for my tasks, it’s not unlikely that it might not have helped me as much as I anticipated or maybe even hampered my efforts,” Philipp Burckhardt, a participant within the examine, wrote in a weblog submit about his expertise.
Why AI is slowing some staff down
So the place did the hares veer off the trail? The skilled builders, within the midst of their very own initiatives, seemingly approached their work with loads of further context their AI assistants didn’t have, that means they needed to retrofit their very own agenda and problem-solving methods into the AI’s outputs, which in addition they spent ample time debugging, in response to the examine.
“The majority of developers who participated in the study noted that even when they get AI outputs that are generally useful to them—and speak to the fact that AI generally can often do bits of very impressive work, or sort of very impressive work—these developers have to spend a lot of time cleaning up the resulting code to make it actually fit for the project,” examine creator Rush instructed Fortune.
Different builders misplaced time writing prompts for the chatbots or ready round for the AI to generate outcomes.
The outcomes of the examine contradict lofty guarantees about AI’s capacity to rework the financial system and workforce, together with a 15% increase to U.S. GDP by 2035 and finally a 25% enhance in productiveness. Actually, many corporations have but to see a return on AI investments. An MIT report printed in August came upon of 300 AI deployments, solely 5% achieved fast income acceleration. Solely 6% of corporations absolutely belief AI to run core enterprise practices, in response to a Harvard Enterprise Overview Analytic Providers analysis report printed final month.
However Rush and Becker have shied away from making sweeping claims about what the outcomes of their examine imply for the way forward for AI.
For one, the examine’s pattern was small and non-generalizable, together with solely a specialised group of individuals to whom these AI instruments had been model new. The examine additionally measures expertise at a selected second in time, the authors stated, not ruling out the chance that AI instruments might be developed sooner or later that may certainly assist builders improve their workflow.
The aim of the examine was, broadly talking, to pump the brakes on the torrid implementation of AI within the office and elsewhere, acknowledging extra information about AI’s precise results should be made recognized and accessible earlier than extra choices are made about its functions.
“Some of the decisions we’re making right now around development and deployment of these systems are potentially very high consequence,” Rush stated. “If we’re going to do that, let’s not just take the obvious answer. Let’s make high-quality measurements.”
AI’s broader affect on productiveness
Economists have already asserted that METR’s analysis aligns with broader narratives on AI and productiveness. Whereas AI is starting to chip away at entry-level positions, in response to LinkedIn chief financial alternative officer Aneesh Raman, it could provide diminishing returns for expert staff corresponding to skilled software program builders.
“For those people who have already had 20 years, or in this specific example, five years of experience, maybe it’s not their main task that we should look for and force them to start using these tools if they’re already well functioning in the job with their existing work methods,” Anders Humlum, an assistant professor of economics on the College of Chicago’s Sales space College of Enterprise, instructed Fortune.
Humlum has equally carried out analysis on AI’s affect on productiveness. He present in a working examine from Might that amongst 25,000 staff in 7,000 workplaces in Denmark—a rustic with related AI uptake because the U.S.—productiveness improved a modest 3% amongst staff utilizing the instruments.
Humlum’s analysis helps MIT economist and Nobel laureate Daron Acemoglu’s assertion that markets have overestimated productiveness features from AI. Acemoglu argues solely 4.6% of duties inside the U.S. financial system can be made extra environment friendly with AI.
“In a rush to automate everything, even the processes that shouldn’t be automated, businesses will waste time and energy and will not get any of the productivity benefits that are promised,” Acemoglu beforehand wrote for Fortune. “The hard truth is that getting productivity gains from any technology requires organizational adjustment, a range of complementary investments, and improvements in worker skills, via training and on-the-job learning.”
The case of the software program builders’ hampered productiveness factors to this want for important thought on when AI instruments are applied, Humlum stated. Whereas earlier analysis on AI productiveness has checked out self-reported information or particular and contained duties, information on challenges from expert staff utilizing the expertise complicate the image.
“In the real world, many tasks are not as easy as just typing into ChatGPT,” Humlum stated. “Many experts have a lot of experience [they’ve] accumulated that is highly beneficial, and we should not just ignore that and give up on that valuable expertise that has been accumulated.”
“I would just take this as a good reminder to be very cautious about when to use these tools,” he added.
A model of this story initially printed on Fortune.com on July 20, 2025.
Extra on AI adoption within the office:
Be part of us on the Fortune Office Innovation Summit Might 19–20, 2026, in Atlanta. The following period of office innovation is right here—and the previous playbook is being rewritten. At this unique, high-energy occasion, the world’s most progressive leaders will convene to discover how AI, humanity, and technique converge to redefine, once more, the way forward for work. Register now.
