Introduction

Artificial intelligence has become a daily companion for developers seeking faster code. Yet the expected productivity gains often hide a more complex reality. A 2025 METR study uncovered a striking gap between how developers feel and what actually happens when AI enters the workflow.

What Happened

In a randomized controlled trial 16 experienced open-source developers completed 246 real tasks in their own codebases with AI tools either allowed or forbidden for each assignment. Participants predicted a 24% speedup and still believed AI had saved them about 20% after finishing. The measured result? They took 19% longer with AI enabled. The study highlights a persistent perception gap that psychology has long documented.

Why This Matters

The core issue isnt that AI writes bad code. It is that humans are predictably poor at supervising something that is usually right and always confident. When code appears instantly the brain registers momentum as progress even if the actual time spent prompting reading correcting and re-prompting tells a different story. Psychologists call this the fluency heuristic things that process easily feel like they are advancing faster. Several well-documented cognitive traps compound the problem. Automation bias leads developers to over-trust AI output especially when the tool is correct most of the time. Vigilance decrement means humans grow worse at spotting rare errors during sustained monitoring. The IKEA effect makes developers overvalue code they have helped prompt into existence making it harder to discard flawed output. Together these effects create a supervision gap that no version upgrade can fully close.

Key Takeaways

  • Measure, dont trust your gut. Track cycle time review time and rework rate for AI-assisted changes. If you cant measure it you are relying on the exact instinct the study showed can be wrong.
  • Write tests before you code. Even a few tests convert the feeling that it looks right into something checkable and turn passive review into active verification.
  • Keep diffs small. Review quality collapses as diff size grows with or without AI. Break work into manageable chunks.
  • Make the model argue against itself. Ask what would break this output then verify those claims. This forces the AI to surface weaknesses you might otherwise miss.
  • Own the merge. Whoever approves a change is responsible for it full stop. Treat AI output as a draft not a final product.
  • Protect your foundational skills. Dont let core coding instincts decay. Write something by hand regularly especially in areas where you are responsible for correctness and ask the model to explain its reasoning not just its output.
  • Apply a sunk-cost rule. If three rounds of prompting have not converged stop. Write the core by hand and use the model only for the boring parts around it.

Conclusion

The true failure mode of AI-assisted engineering isnt mostly that the model writes bad code. It is that humans are predictably bad at supervising something that is usually right and always confident. Tools will keep improving but the human side of the loop does not get a version upgrade. Teams that get real value wont be the ones with the best model they will be the ones who designed their process around how people actually behave not relying on gut feeling.

Have you measured your own AI-assisted cycle time or are you going on feel? I would like to hear about what you found in the comments.