The Hype vs. The Data
I have spent the last few years watching engineering teams chase the latest productivity silver bullet. First it was Kubernetes, then platform engineering, and now AI-assisted coding. Every trend arrives with a promise of faster delivery, happier developers, and better software. But the 2024 DORA report, which aggregated responses from nearly 3,000 tech workers, tells a different story. 1
Nathen Harvey, DORA team lead at Google Cloud, put it bluntly: engineers using AI report higher individual productivity and better flow, yet those gains are not translating into faster or safer delivery. 2 In fact, the data shows that increased AI adoption is actively detrimental to core software delivery performance metrics.
AI Adoption: The Productivity Paradox
The numbers are striking. The 2024 DORA report found that an increase in AI adoption reduces delivery stability by 7.2% and delivery throughput by 1.5%. 2 That is not a rounding error; it is a measurable regression in the very metrics we optimize for.
There are some bright spots. A 25% increase in AI adoption correlates with a 3.4% improvement in code quality and a 3.1% faster code review speed. 2 Documentation also gets a 7.5% boost, which is significant because the 2023 DORA report found that quality internal documentation has an almost 13-fold impact on organizational performance. 2
But here is the catch: those upstream gains are not flowing downstream. AI helps us write more code at the start of the lifecycle, yet it does not solve the later stages—testing, deployment, and operations. 2 More code without better test automation means larger batches, harder reviews, and less stable releases. It is the classic bottleneck shift, not a bottleneck removal.
This disconnect is also visible in developer sentiment. The Atlassian-DX State of DevEx Survey found that two-thirds of developers have yet to see any significant AI productivity gains, even though leadership believes AI is the most effective way to improve productivity. 3 More than a third of DORA respondents reported little to no trust in AI-generated code. 2 When I read that, I nodded. I have seen teams merge AI-generated pull requests only to spend the next sprint debugging corner cases the model never considered.
Platform Engineering: A Similar Story
Platform engineering follows a nearly identical pattern. The 2024 DORA report found that having a dedicated platform engineering team increases productivity at the software development team level by 6%. 2 That is a solid win on paper.
Yet the same focus on platform engineering does not improve core DORA metrics. Throughput drops and change failure rates increase. 2 The DORA team hypothesizes that internal developer platforms add handoffs between systems and teams, and that the increased experimentation they enable leads to more failed changes. 2
I have lived this. A few years ago, I watched a team build an ambitious internal platform that abstracted away deployment complexity. Developers loved it—until they needed to debug a failure in the abstraction layer. The platform had become a black box, and the cognitive load it added offset the time it saved.
The report also notes that platform-induced instability correlates with higher developer burnout. 2 That is a sobering reminder: a platform meant to reduce toil can instead create stress if it is not treated as a product with tight feedback loops to its internal users.
What Actually Moves the Needle
If AI and platform engineering are not the answers, what is? The DORA data points to two old-fashioned practices that still dominate.
First, documentation. The 2023 DORA report showed that quality internal documentation has an almost 13-fold impact on organizational performance. 2 The 2024 report reinforces this: AI can help write docs, but the docs themselves are what matter. Teams with good documentation move faster because they spend less time searching for answers and more time shipping.
Second, user-centricity. The 2023 report found that a greater user-centricity led to a 40% increase in organizational performance. 2 This year’s report confirms that organizations prioritizing the end-user experience not only produce higher quality products, but their developers are more productive, more satisfied, and less likely to experience burnout. 2
These are not tooling problems. They are cultural problems. And culture is harder to buy than a GitHub Copilot subscription or an internal developer platform.
Smaller Batches, Tighter Loops
Harvey’s advice from the report resonates with me: the way to improve software delivery performance is to ship smaller changes. 2 AI helps us write more code than ever, which tempts us into larger pull requests. Larger batches are harder to review, slower to test, and riskier to deploy. If we let AI generate bigger changes without tightening our feedback loops, we are trading local speed for global instability.
I have started asking my teams a simple question: are we shipping smaller, safer changes than we were six months ago? If the answer is no, no amount of AI or platform investment will fix what is broken.
The Bottom Line
The 2024 DORA report is a reality check. AI and platform engineering are powerful tools, but they are not silver bullets. They improve individual productivity and team satisfaction while harming the core metrics that define delivery excellence. The real levers—documentation, user-centricity, smaller batches, and a culture of continuous improvement—require patience and discipline, not vendor invoices.
As an industry, we are likely at the bottom of the J curve with both AI and platform engineering. 2 The hype is ahead of the impact. My takeaway is simple: invest in the basics before you chase the shiny objects. The data is on your side.
References
[1] Google Cloud DORA, “2024 Accelerate State of DevOps Report,” 2024. https://dora.dev/research/2024/dora-report/
[2] Jennifer Riggins, “Dora 2024: AI and Platform Engineering Fall Short,” The New Stack, 2024. https://thenewstack.io/dora-2024-ai-and-platform-engineering-fall-short/
[3] Atlassian and DX, “2024 State of Developer Experience Survey,” 2024. https://www.atlassian.com/software/compass/resources/state-of-developer-2024