The Irreplaceability Trap

Cowan was judging the explanation, and the account in Google’s SRE book doesn’t measure its effect on reliability. The new engineer had already learned investigation techniques in a training class. She brought the experts an explanation to check, rather than asking them to supply it.

Who Gets to Practice

Suppose a blocked engineer sends every unfamiliar production question to the person who knows the service best. The expert answers quickly. If the answer arrives as a fix to copy, the colleague can complete the task without choosing what to inspect or rejecting a wrong explanation. The next urgent request gives both people a reason to repeat the arrangement.

During an outage, taking the expert’s safe fix is reasonable. The cost appears when ordinary releases and investigations follow the same route: the next owner gets no place to practice deciding before a customer is waiting.

Clear procedures are enough for some gaps. If a colleague already understands when a rollback is safe but lacks the command or permission to run it, fix the procedure or access and watch them use it. Where they still need the expert to decide what the evidence means, a documentation assignment alone leaves that dependence intact.

Dependence alone tells you little about motive. A founder may have built the service before there was anyone to teach. An engineer may refuse to share what they know. Those call for different conversations, but either way, replacing the person while leaving the work arranged around a sole expert preserves the risk.

Leadership has to decide which delivery work will wait while someone else learns. Both people need room for the handoff. Giving them that assignment while keeping their other commitments unchanged makes the cost theirs to absorb.

Give the Next Owner the Controls

Choose a bounded responsibility, such as deploying a service and recovering from a bad release. Start with the existing runbook. Have the future backup revise it as they work through the task, marking the points where they had to guess. The expert reviews those gaps, including steps that looked too obvious to need explaining.

Use a staging system or an isolated copy for the practice run. Suppose the service starts returning errors after a release. The rollback procedure gives the learner a command; choosing it requires deciding whether the release is a plausible cause and whether reverting it is safe.

For this rehearsal, compare how the old and new versions handle the same test requests against the same dependencies. Failure confined to the new version makes the release a stronger suspect. If both versions fail at the same dependency, a rollback may leave the failure untouched. Ask the learner what they’d inspect next and what result would change their mind.

If they choose a rollback, have them check its assumptions before acting. Can the previous version still use the current database? After reverting, does the request that failed now succeed? The rollback command belongs in the procedure; so does the evidence for choosing it.

Google’s book describes a later stage of training called reverse shadowing. Some teams let the new engineer take primary on-call responsibility while an experienced engineer diagnoses independently, without changing the system. The experienced engineer remains available to provide help and hints.

Borrow that division of responsibility for the task you’ve rehearsed, within the learner’s demonstrated readiness. Agree beforehand on the scope of permitted changes and when the expert will step in. Restoring service takes precedence over finishing a lesson.

Record the help given. If the expert supplies the diagnosis, the learner may have demonstrated command execution while the diagnostic handoff remains unfinished. An intervention can also expose missing access or authority to act, which calls for a different response than more practice.

Address the gap, then repeat the task with a changed condition. If the rehearsal always rewards rollback, a memorized command can look like judgment. A colleague making and explaining the decision, carrying it out, and checking the result gives you evidence for that particular responsibility.

Keep the boundary honest. Recovering from a rehearsed failure doesn’t establish readiness for every unfamiliar outage. The expert may remain the right escalation point for difficult cases. If there’s no second operator available to train, the business has a staffing constraint to resolve; writing more pages can’t supply the missing person.

What the Expert Gets Credit For

Your strongest engineer can retain deep expertise while colleagues become capable of operating the service. The useful change is that routine decisions no longer have to wait for them. They can spend their attention on the cases that still require it.

That only becomes a fair expectation if helping colleagues develop judgment counts as engineering work. A review based entirely on personal feature output and incident rescues gives the expert little credit for the handoff you asked them to lead.

At the next review, ask that engineer to bring a change somebody else led. Let the colleague explain why they chose it, how they checked it, and where they still needed help.