The change request said “Rollback: reverse the above steps.” Four words. It sailed through our change advisory board on a Tuesday, the same way dozens of nearly identical requests had sailed through before it, and at about 2:40 the following Sunday morning an engineer on my team discovered that the above steps could not be reversed. The configuration he had pushed to a core routing pair had overwritten the state he needed to get back to. The outage ran a little under three hours. Customers noticed. So did my boss.

The post-incident review went the way these things usually go at first. Who approved it? Who wrote it? Why didn’t anyone catch it? And then somebody pulled the last year of change records, and the room got quiet. Almost every rollback plan in the file read like that one. A year earlier our standard had required a tested, step-by-step back-out procedure with a named owner. Nobody had ever voted to drop it. Nobody had announced that “reverse the steps” was now acceptable. It had simply become what a rollback plan looked like, one approved shortcut at a time, and every one of those approvals had been followed by a maintenance window that went fine.

That is the thing I want to walk through here, because I have seen the same pattern since then in warehouses, finance teams and a dental practice, and it almost never gets caught by the people living inside it.

How a rollback plan turned into one sentence

Reconstructing it afterwards was uncomfortable, mostly because there was no villain.

The original rule existed for good reasons, and the early requests followed it. Then came a busy quarter. A senior engineer, one of the best I have ever worked with, submitted a low-risk change with an abbreviated rollback because writing the full procedure would have taken longer than the change itself. The board approved it; it was obviously low risk, and he obviously knew what he was doing. The change went fine.

The next person who looked at the change records for an example saw his. Their change was a little less trivial, and the rollback a little thinner. Approved. Fine. Within a few months, the abbreviated version was the template people copied, the board was reviewing forty requests a week instead of twenty, and the question in the room had shifted from “is this rollback adequate?” to “has this engineer done this kind of thing before?” Every individual decision was defensible. The sum of them was a team that had quietly stopped doing back-out planning while still believing it had a back-out process.

The part that still bothers me is that I was on that board. I read those requests. I did not see the drift, because each one only looked slightly different from the one before it, and each one before it had worked.

NASA had a name for this before I did

The sociologist Diane Vaughan spent years inside NASA’s records after the Challenger disaster, and she came out with a term for exactly what happened to our rollback plans: normalization of deviance.

Her starting assumption, she has said, was that she would find misconduct: managers knowingly breaking rules under schedule pressure. What she found instead was more ordinary and more frightening. The solid rocket booster O-rings had shown damage on earlier shuttle flights, which was not supposed to happen at all. Each time a damaged booster came back and the mission had succeeded anyway, the anomaly was reviewed and, in her words, the “technical deviation of the [O-rings] from performance predictions was redefined as an acceptable risk.” Repeated often enough, that redefinition became a script. People “developed a definition of the situation that allowed them to carry on as if nothing was wrong,” as the University of Chicago Press summarized her findings. On January 28, 1986, Morton Thiokol engineers who had recommended against launching below 53°F watched Challenger lift off at 36°F, and it broke apart 73 seconds later. Vaughan’s conclusion was that “mistakes themselves are socially organized and systematically produced.”

Seventeen years later NASA did it again. A 1.7-pound piece of insulating foam struck Columbia’s left wing during its January 2003 launch. Foam shedding from the external tank had happened on previous flights; the specifications said it should not happen at all, and yet it had been reclassified, flight by flight, as a maintenance issue rather than a safety one. The Columbia Accident Investigation Board found a “broken safety culture” and listed “reliance on past success as a substitute for sound engineering practices” among its causes, according to the Congressional Research Service’s synopsis of the report. Its blunt verdict: “the causes of the institutional failure responsible for Challenger have not been fixed.”

If the most rigorous engineering organization on earth could drift twice, with the whole world watching, your team’s expense approvals, safety checks and code reviews are not immune.

Why capable teams drift

The uncomfortable lesson from both NASA and my change board is that normalization of deviance is not a discipline problem. It happens to competent people precisely because they are competent enough to get away with the shortcut.

John Banja, an ethicist at Emory, studied the same pattern in hospitals and published a breakdown of why it persists in Business Horizons in 2010. His point is that violations begin as deviations and, “with enough repetitions,” become normal practice patterns until “personnel no longer regard these acts as untoward.” Several of his drivers map almost perfectly onto any operations team I have run:

• “The rules are stupid and inefficient.” When compliance feels like friction with no visible payoff, people build workarounds. Our full rollback procedure genuinely was overkill for some changes, and that made it easy to treat as overkill for all of them.
• Knowledge is uneven. New people learn the deviation as the standard. Half my team had joined after the shortcut became normal; they had never seen the original rule followed.
• “The rules don’t apply to me.” Experienced people trust their own judgment over the procedure, and they are usually right, which is what makes the exceptions so persuasive to everyone watching them.
• People are afraid to speak up. A junior engineer told me after the outage that she had wondered about the thin rollback plans for months. She assumed the board knew something she didn’t.
• Leaders soften bad news on its way up. Small near misses get summarized as “minor issue, resolved” and never reach anyone who could see the pattern.

Underneath all of it sits an idea the physician and safety researcher Richard Cook put better than anyone in his short paper How Complex Systems Fail: “complex systems run as broken systems.” There are always latent flaws in the way work actually gets done. Success does not prove they are absent; it only proves they have not lined up yet. Every uneventful maintenance window was teaching us the wrong lesson.

What we changed, and what it cost

We did not fix it by tightening the rule and yelling louder. That approach tends to produce better paperwork and the same behavior. Here is what actually worked, and what I still use with clients in my fractional COO work.

We made the standard fit the risk. The original rule’s one-size-fits-all requirement was part of why people abandoned it. We split changes into three risk tiers. Low-risk changes got a short, honest rollback. High-risk changes required a back-out procedure that had been executed in the lab, with the evidence attached. Fewer rules people resented meant fewer rules people quietly ignored.

We compared current practice to the written standard on a schedule, not after an incident. Once a quarter, someone outside the team pulled ten random change records and read them against the policy as written, not against last month’s records. That is the only comparison that catches drift, because comparing this month to last month always says “about the same.” It is the operational version of the checklist discipline that matters most on the work everyone thinks they know cold.

We treated a good outcome as a separate question from a good decision. In reviews, “did it work?” and “did we follow the plan we agreed to?” became two different questions with two different answers. A change that succeeded with no real rollback plan was logged as a near miss, not a win. That single habit did more than anything else, because it stopped success from silently rewriting the standard.

We rewarded the person who asked the awkward question. The junior engineer who had wondered about the rollback plans ran the next quarterly sample. That told the whole team, more clearly than any memo, that noticing drift was part of the job. If your team goes quiet in exactly these moments, the problem is upstream, in whether it is safe to speak up at all.

We did the blameless review properly. The engineer who pushed the change was not the cause; he was the person standing where the drift finally hit the ground. A root cause analysis that had stopped at him would have left every other thin rollback plan sitting in the queue.

The cost was real. Change throughput dropped for about two months while people relearned how to write back-out procedures, and a couple of senior engineers grumbled loudly that we were treating them like trainees. Then it leveled off, and the following year we had zero outages caused by failed rollbacks. I would make that trade again without thinking about it.

If you want one thing to take from this, it is a question to ask about any process your team has run successfully for a long time: if the person who wrote the original standard watched us do this today, would they recognize it? When the honest answer is “probably not,” you are not looking at a team that has gotten efficient. You are looking at a team that has been lucky, and has started to mistake the luck for the process.