The Judgment Reserve

Suppose your senior engineer hands a failing test to an AI assistant and gets a patch back. The test passes; the ticket closes. If the engineer never works through why the original code failed, the task is complete without that engineer having practiced the diagnosis.

For a familiar defect, that may be a sensible trade. Apply it to unfamiliar problems as well, though, and you have a question to answer: where does the engineer get to practice the reasoning you’ll still expect when the suggested fix fails?

Lisanne Bainbridge described this problem in Ironies of Automation in 1983. Writing about industrial control systems, she pointed out that operators could lose opportunities to maintain the skills needed to take over when automation failed. Her discussion of cognitive skills was specific: retrieving knowledge depends on using it, and practical understanding develops through use and feedback. Leaving a person responsible for exceptions doesn’t, by itself, give them that practice.

The 2025 Your Brain on ChatGPT preprint, from researchers including MIT Media Lab’s Nataliya Kosmyna, reported results from 54 adults recruited at Boston-area universities, most of them students. Participants wrote essays with ChatGPT, a search engine, or no tools. The researchers reported weaker connectivity in EEG recordings of brain activity and more difficulty quoting their essays among ChatGPT users than among unaided writers. Those findings concern a small essay-writing study; they don’t establish lasting loss of engineering skill, or show that senior engineers become less capable whenever they use AI.

There is also evidence of thinking moving rather than disappearing. In a 2025 survey of 319 knowledge workers, researchers at Microsoft and Carnegie Mellon found that higher confidence in AI was associated with less reported critical thinking. But respondents also described work shifting toward checking information and deciding how to use the output. These were self-reports, with no test of long-term skill retention.

The risk we infer for engineering is a mismatch between responsibility and practice. If you continue relying on someone’s judgment while removing their occasions to exercise it, you’re making a bet on retained skill. Call that accumulated understanding and practiced reasoning the judgment reserve: the capacity you call on when the answer needs more than a familiar recipe.

A count of closed tickets won’t settle the question. The unexamined patch and the carefully diagnosed one both count as a closed ticket. An unchanged patch doesn’t prove passivity either; the engineer may have evaluated it and agreed. If delegation frees them to investigate a difficult assumption, they’ve gained an occasion for judgment. Less typing, or less reported effort, can’t tell you which of those happened.

Which friction is waste, and which friction is training? Hand off the repetitive code for a familiar migration, gladly. Keep the work of deciding what data it must preserve and how you’d recover if it stops halfway. The boilerplate for an endpoint that creates or updates records can be delegated too; deciding who may change those records still needs an answer grounded in the actual access rules. The same ticket can contain both kinds of work.

Suppose the failing test concerns a migration that copies records and saves a checkpoint marking its progress. The suggested patch passes a test that interrupts the job just after the checkpoint is saved. If the copy commits before the checkpoint does, though, a crash in between leaves the data ahead of the saved progress. Resuming from that checkpoint will process some records again.

Whether that repeat is harmless depends on the schema and what the write does. It might create duplicates or stop at a uniqueness constraint. An engineer can predict the effect, interrupt the job at that boundary in a disposable database, and inspect the records after restart. AI can write every line of that experiment, or suggest a competing diagnosis. The practice lies in deciding what observation would distinguish the explanations, then following the evidence when it contradicts the favored one.

For the next unfamiliar change, ask its reviewer what would make the proposed fix fail and how they checked. A polished explanation can come from the model too, so asking them merely to repeat it gives you little evidence of their understanding. In the migration example, move the interruption point and ask what changes in the expected result. If they can’t account for the difference, there’s a specific piece of reasoning to work through together before you merge.

That exchange can show how someone reasons about this case. It can’t establish how well they’ll retain the skill or handle a different system. The cited AI studies didn’t test these review exercises as a remedy for engineering skill loss.

This costs attention that could have gone into the next ticket. Spend it on reasoning the team will need again, rather than requiring people to type familiar code from memory. An existing review or a replay with a colleague can provide the practice; a separate no-AI exercise would need a reason of its own.

During an incident, use the safest recovery available. A replay after recovery can provide the place to investigate the parts someone else solved, without making a customer wait for the lesson.