- Experienced developers actually spent 19% more time in tasks when using AI, the report states
- Only 44% of the code generated by AI was accepted by experienced developers
- Developers still feel that work is easier when used to
Artificial intelligence may not be so beneficial for experienced developers such as new entrants and those who have not yet developed the appropriate skills, according to new research.
A new study by the evaluation of the model and threat research (METR) has suggested that developers were not only less optimistic after study, but that real results suggest that artificial intelligence actually ended up costing them time.
The study found that, with AI, the developers spent less time encoding and looking, and more time, waiting and more important, reviewing, the production of AI. It is estimated that 9% of the time continued to review and clean the code generated by AI, with suggestions of AI generally on the right tracks, but lacks details.
Ai is not really equal to these developers at any time
Observing 16 developers experienced in 246 real tasks on ripe open source projects with which they were already familiar, the researchers analyzed how the developers interacted with popular tools of Cursor Pro and Claude 3.5/3.7.
Before the study, the 16 experienced developers in question expected to reduce the task time by 24% by combining their experience with artificial intelligence. After the study, they reduced their expectations at 20%, however, the analysis after the study reveals that AI actually increased the task completion time by 19%.
Less than 44% of suggestions were accepted, with a lack of contextual knowledge and large and complex repositories as contributing factors to the developer’s deceleration. The study also pointed out that experienced developers already had a great familiarity with the code bases, leaving little space for AI to add any significant value.
However, despite the deceleration, many developers continued using AI tools because the work felt less effort, making work feel more pleasant even if it was not faster.
“The capacities of the nature can be lower than the results at common use points can suggest,” concludes research work.
