An update from the study that said devs were actually slower with coding agents

METR: We are Changing our Developer Productivity Experiment Design

Based on conversations with study participants, we believe it is likely that developers are more sped up from AI tools now — in early 2026 — compared to our estimates from early 2025. However, because of the selection effects in our experiment, our data is only very weak evidence for the size of this increase.

Remember that study last year that showed that developers who used AI tools were 20% slower than those who didn’t? Of course you do, it was all over the place!

This update from back in February that I just found today is about how they had to change their methodologies for testing because so much had changed. Additionally, while they don’t have conclusive results, their early findings seem to indicate that the uptick in productivity is actually pretty significant now.

For some context, if you haven’t followed this closely, their original study was of developers using Claude Sonnet 3.5 through cursor while this new study would’ve been using Claude Code with Opus 4.5 anyone who’s worked with this tooling at all knows that that is night and day.