Comprehension debt

Why it matters

A team carrying comprehension debt responds slowly when something goes wrong. Code that nobody understands can run for months without trouble. Then a serious incident, a security fix or a change in requirements arrives, and someone has to reverse-engineer the code under time pressure before they can touch it. AI coding tools make the debt easier to build up because they separate two things that used to happen together: producing code and understanding it. A developer who wrote a module by hand had to reason through it line by line. The same module can now arrive in minutes, pass its tests and be merged by someone who has read only part of it.

Code review used to be where knowledge of new code spread across a team. As Addy Osmani points out, AI generates code far faster than people can evaluate it, so a junior developer can now produce changes faster than a senior developer can audit them. Review stops being a quality gate and becomes a throughput problem.

What is comprehension debt?

Comprehension debt is the growing gap between how much code exists in a system and how much of it anyone can explain. Addy Osmani described it in a blog post on 14 March 2026, “Comprehension Debt – the hidden cost of AI generated code”, and O’Reilly Radar republished the piece on 13 April 2026. His post links to earlier work by Margaret-Anne Storey, professor of computer science at the University of Victoria, who wrote about “cognitive debt” in February 2026. Storey draws on Peter Naur’s argument that a program is a theory held in the minds of the people who build it: what the software does, why it was designed that way and how it can change. When that shared theory falls apart, the team loses the ability to change the code safely, even if the code itself is clean.

How is comprehension debt different from technical debt?

Technical debt sits in the code, while comprehension debt sits in the people who maintain it. Ward Cunningham coined the technical debt metaphor in 1992 for the future cost of choosing an expedient solution now, and Martin Fowler describes the interest on it as the extra effort each new feature takes because of poor internal quality. Technical debt makes itself felt. Builds slow down, dependencies tangle and every change to a certain module feels risky. The comprehension gap builds up without that friction. The code can look tidy and pass every test while the team’s grasp of it weakens, which Osmani says breeds false confidence. Storey draws the same line for cognitive debt, which lives in developers’ minds, and a 2026 study from Karlstad University in Sweden places comprehension debt in the collective cognition of a development team. The two debts can arrive together. AI-generated code that duplicates existing logic adds technical debt, and if nobody reviewed it closely, it adds comprehension debt as well.

Why delivery metrics miss comprehension debt

Standard engineering metrics count output and flow. Osmani notes that velocity, DORA delivery metrics and test coverage can all look healthy while comprehension falls behind, because a typical measurement system has no way to tell whether anyone can explain the code. A merged pull request counts the same whether its reviewer understood it or skimmed it.

Google Cloud’s 2025 DORA report, based on responses from nearly 5,000 technology professionals, found that AI adoption now has a positive relationship with software delivery throughput but still a negative one with delivery stability. The report’s explanation is that faster development exposes weaknesses further down the delivery chain. DORA did not measure comprehension, so the finding is not proof of comprehension debt, but it fits the pattern of teams shipping more and paying for it later. Shopify’s engineering leadership, quoted by Bessemer Venture Partners in June 2026, uses weekly demos as its main signal for comprehension debt. A demo shows whether a team understands what it has built, which revert rates alone do not.

Research on AI and developer understanding

The clearest experimental evidence so far comes from a study by Judy Hanwen Shen and Alex Tamkin of Anthropic, posted on arXiv in January 2026. In a randomised experiment, 52 participants, most of them working programmers, learned Trio, an asynchronous Python library, with or without an AI assistant. The AI group scored 17% lower on the follow-up quiz, about two grade points (Cohen’s d = 0.738, p = 0.010), and did not finish significantly faster. The quiz covered conceptual understanding, code reading and debugging, the skills a developer needs to supervise AI-written code.

How people used the AI changed the result. Participants who delegated the task, leaned on the AI more and more, or used it to debug by trial and error scored less than 40%. Those who asked conceptual questions, asked for explanations alongside code, or generated code and then worked through it scored 65% or more. The groups were small and the paper is a preprint, so the figures are an early signal.

A second 2026 preprint, by Muhammad Ovais Ahmad of Karlstad University, analysed 621 reflective diaries written by 207 undergraduate software engineering students during an eight-week Scrum project. It found four ways the debt built up: accepting AI output as a black box, mismatches between generated code and project context, skills weakening through dependence on AI, and skipped verification. One pattern worked against it: students who used AI to build their own understanding of the code.

How do you recognise comprehension debt?

The early signs are in how people behave around the code. Storey names hesitation to make changes, knowledge concentrated in one or two people, and parts of the system that the team treats as a black box. Simon Willison described the individual version in February 2026. After prompting whole features into existence without reviewing their implementations, he no longer had a firm mental model of what his own projects could do, and each new feature became harder to reason about.

How can a team prevent comprehension debt?

A team prevents comprehension debt by making understanding part of the definition of done. The sources above suggest these practices:

• Make sure at least one person fully understands each AI-generated change before it reaches production (Storey).
• State what a change is supposed to do before generating it, so the reviewer can check the code against that intent (Osmani).
• Record why a decision was made as well as what changed (Storey).
• Ask the AI to explain code and concepts as well as write them. In Shen and Tamkin’s experiment, the patterns built on questions and explanations kept learning intact.
• Keep engineers’ knowledge two or three layers below the level they work at. Farhan Thawar, head of engineering at Shopify, puts the rule as handing the toil to AI and keeping the thinking.
• Hold regular demos and walkthroughs where the team explains its own system out loud.
• Slow down where it counts. A breakout group at the Future of Software Engineering Retreat, organised by Martin Fowler and Thoughtworks, concluded that pair programming, refactoring and test-driven development reduce cognitive debt as well as technical debt (Storey).
• Plan review capacity alongside code generation, since faster generation lands on the reviewers first.

Frequently asked questions

What is comprehension debt in software development?

Comprehension debt is the gap between the code a team has shipped and the code it understands. It builds up when changes, often AI-generated ones, are merged faster than people can read and reason about them. The cost appears later, when someone has to fix or extend code that nobody can explain.

Where does the term comprehension debt come from?

It was described by Addy Osmani in a March 2026 blog post, which O’Reilly Radar republished a month later. The post builds on Margaret-Anne Storey’s February 2026 writing on cognitive debt, which in turn draws on Peter Naur’s view of programming as building a shared theory of a system.

Is comprehension debt the same as technical debt?

No. Technical debt is a quality problem in the code and usually shows up as friction: slow builds, tangled dependencies, risky changes. A team’s understanding can erode while the code looks clean and every test passes, and that erosion is comprehension debt. A codebase can carry both.

What is the difference between comprehension debt and cognitive debt?

The terms are often used for the same problem, and Osmani’s post calls it “comprehension debt or cognitive debt”. Storey’s framing of cognitive debt stresses the team’s shared theory of why the system is built the way it is. Osmani’s definition measures that understanding against the amount of code in the system and puts the speed of AI code generation at the centre.

Does AI cause comprehension debt?

AI speeds it up when developers delegate instead of engaging. In Shen and Tamkin’s 2026 experiment, developers who learned a new library with AI help scored 17% lower on a comprehension quiz. Participants who used the AI to ask conceptual questions and get explanations scored 65% or more, against under 40% for those who handed the task over.

How do you measure comprehension debt?

There is no standard metric for it, and velocity, DORA metrics and test coverage do not capture it. The practical checks are qualitative. Someone on the team should be able to explain a given change, walk through a module in a demo and diagnose an incident without first reverse-engineering the code.

Sources

• Addy Osmani (2026): Comprehension Debt – the hidden cost of AI generated code – blog post of 14 March 2026; comprehension debt as the gap between the code that exists and the code people understand; false confidence, the review speed asymmetry and the measurement gap (velocity, DORA metrics, coverage).
• Addy Osmani, O’Reilly Radar (2026): Comprehension Debt: The Hidden Cost of AI-Generated Code – republication of the blog post on 13 April 2026.
• Margaret-Anne Storey (2026): How Generative and Agentic AI Shift Concern from Technical Debt to Cognitive Debt – University of Victoria; cognitive debt lives in developers’ minds; Naur’s program-as-theory; mitigation practices and warning signs.
• Simon Willison (2026): How Generative and Agentic AI Shift Concern from Technical Debt to Cognitive Debt (link post) – first-hand account of losing the mental model of his own AI-built projects.
• Shen & Tamkin (2026): How AI Impacts Skill Formation – randomised experiment, 52 participants learning the Trio library; AI group scored 17% lower on the quiz (Cohen’s d = 0.738, p = 0.010) with no significant speed-up; six interaction patterns. Preprint.
• Ahmad (2026): Comprehension Debt in GenAI-Assisted Software Engineering Projects – Karlstad University; 621 reflective diaries from 207 undergraduate students; four debt-accumulation patterns and one mitigating pattern. Preprint.
• Google Cloud / DORA (2025): Announcing the 2025 DORA Report – nearly 5,000 respondents; AI adoption positively related to delivery throughput, negatively to delivery stability.
• Bessemer Venture Partners (2026): Inside AI-pilled engineering teams: Five lessons for scaling without losing the plot – Shopify’s Farhan Thawar on comprehension debt, weekly demos and keeping knowledge two to three layers deep. Investor publication.
• Martin Fowler: Technical Debt – principal and interest; interest as the extra effort new features take because of poor internal quality.
• Wikipedia: Technical debt – metaphor introduced by Ward Cunningham in 1992; cost of choosing an expedient solution.