Doing Something Felt Better. It Made the Outage Twice as Long.


timelapse photo of soccer player kicking ball

Forty minutes into a regional service outage, a director dialed into the bridge call and asked me a simple question: “What are we doing right now?”

The honest answer was “watching.” My team had rerouted traffic away from the failing platform, customers were degraded but mostly back in service, and the vendor’s engineer was reading logs to figure out why a core component had started dropping sessions. Nothing else needed to happen for another twenty minutes or so. But “watching” is a hard word to say to a director at two in the morning, so when one of my senior engineers suggested we restart the component, I said yes.

The restart cleared the logs the vendor was reading. It also kicked off a resynchronization that took the rerouted traffic down with it. A degraded outage became a full one, and the root cause investigation started over from scratch. We were down for roughly twice as long as we would have been if I had simply said “we are holding, and here is why.”

I ran network operations at a large telecom for a long time, and that night is the one I come back to when I think about my own judgment under pressure. The restart was not a technical mistake so much as a self-management one. I approved it because doing something felt better than doing nothing, and because I wanted an answer to give the person asking.

That impulse has a name.

Goalkeepers dive even when standing still works better

In 2007, a team led by Michael Bar-Eli at Ben-Gurion University published a study of 286 penalty kicks taken in top leagues around the world. The finding, summarized by the British Psychological Society’s Research Digest, is almost comic. Goalkeepers dove left or right on 93.7 percent of kicks. When they stayed in the center, they saved 33.3 percent of shots. When they dove right, they saved 12.6 percent; when they dove left, 14.2 percent.

Staying put was the best strategy on the numbers, and keepers almost never used it. The researchers’ explanation, laid out in the paper itself (Journal of Economic Psychology, 2007), is that a goal conceded while standing still feels worse than a goal conceded while diving. Diving is the norm. If you dive and miss, you did your job and got beaten. If you stand there and the ball goes in, it looks like you did nothing at all. A survey of 32 professional keepers backed this up: jumping was what they believed they were supposed to do.

Swap the goalkeeper for a manager on a bridge call and the logic is identical. If I approve the restart and it fails, I made a reasonable call that didn’t work out. If I hold and the outage drags on, I am the manager who sat there.

Why waiting feels like failure

Anthony Patt and Richard Zeckhauser gave this tendency its working name in a 2000 paper, “Action Bias and Environmental Decisions”. Their framing is fair to the instinct: people have a penchant for action, often for good reasons. Most of the time, acting is how problems get solved. The bias shows up when that penchant gets carried over into situations where the reasons no longer apply.

There is a second force underneath it, and it is less flattering. In a series of studies published in Science in 2014, Timothy Wilson and colleagues left people alone in a room with nothing to do but think for 15 minutes. The room had a button that delivered a mild electric shock, one each participant had already felt and said they would pay money to avoid. Twelve of 18 men and six of 24 women pressed it anyway at least once. The study has its critics, who argue the shock was only mildly unpleasant, but the core result is hard to shake: sitting with your own thoughts and no task is uncomfortable enough that a lot of people will reach for almost anything.

Put those together and you get the manager’s version. Holding a position is uncomfortable to sit in and awkward to explain. Acting relieves both problems at once, whether or not it helps.

My own operations world had numbers on what that costs. Google’s site reliability engineers have written that roughly 70 percent of outages are due to changes in a live system. Every restart, config push, and “let’s just try this” on a bridge call is a change in a live system. During an incident, the urge to act is the urge to introduce more of the thing that causes incidents.

What we changed after that night

I did not fix this by becoming a calmer person. I fixed it, mostly, by changing the structure around the decision so that holding stopped looking like nothing.

We made “hold” a named action. On every major incident bridge after that, the incident lead stated the current posture out loud every 15 minutes: “We are holding on the reroute. Next decision point is 03:10. Here is what would change that.” Holding with a time box and a trigger is a plan. Holding with no explanation is silence, and silence is what makes executives start suggesting things.

Every proposed action had to answer one question. “What does this buy us that waiting doesn’t?” A restart that would restore service while preserving diagnostics was worth doing. A restart that might help and would definitely destroy evidence was not. Most bad incident actions fail this question in about ten seconds, once somebody actually asks it.

I gave the executive a job. The director on the call that night was not being unreasonable; they were being asked by their own boss for an update and had nothing to say. After that incident, we sent leadership a short written status on a fixed schedule. Once they had a predictable answer to “what’s happening,” the pressure for visible activity on the bridge dropped sharply.

We separated reversible moves from irreversible ones. Rerouting traffic could be undone in a minute. Restarting a component that cleared its own state could not. We started treating the second kind like a change request even in the middle of an outage: name it, say what it destroys, and get a second voice before doing it. That distinction is the same one I wrote about in most decisions being more reversible than managers act like they are; in an outage the line matters more, not less.

None of this slowed us down when action was the right answer. It slowed us down only when action was a way of managing our own discomfort.

The quieter versions of the same mistake

Bridge calls make action bias obvious because the clock is loud. Most of the time it hides in ordinary management weeks.

A new manager restructures the team in their first month because they feel they need to show something, before they understand why the current setup exists. (That one has its own trap, the one behind pulling down a fence before you know why it was built.)

A strong performer has one bad quarter, and the manager launches a formal plan when a conversation and a few weeks of patience would have done it.

A metric dips for a month and the response is a new process, a new dashboard, and a new weekly meeting, when the dip was noise.

A conflict between two people on the team is starting to resolve itself, and the manager steps in and makes it about the manager.

In each case, the intervention is easy to defend afterward. Nobody gets criticized for “taking it seriously.” That is exactly why it keeps happening, and why judging these calls by how they turned out is a trap of its own (a good result doesn’t mean the call was good, and neither does a defensible one).

A test I still use on myself

When I feel the pull to step in, I ask two questions before I do anything.

First: if I do nothing for the next hour (or day, or week, depending on the situation), what specifically gets worse? If I can name it, I act. If my honest answer is “nothing, but it would look like I’m not on top of this,” the urge is about me, not about the problem.

Second: what would I tell someone who asked what I’m doing? If the answer is “holding, until X happens or until Y o’clock,” I say exactly that, out loud, to whoever needs to hear it. Most of the pressure to act comes from not having a good sentence for waiting. Write the sentence and much of the pressure goes away.

I still think about that restart. Not because it was a catastrophe (it was a long night and a bad report, nothing worse), but because I knew while I was approving it that I was approving it for the wrong reason. The keeper who stands in the middle of the goal is not doing nothing. They are choosing the position with the best odds and accepting that it will look bad when the ball goes past them anyway. Managers need the same nerve, and it is a skill you can practice, one decision at a time.

Ty Sutherland

Ty Sutherland is an operations and technology leader with 20+ years of experience. He is Director of IT Operations at SaskTel, founder of Ops Harmony (fractional COO and EOS Integrator), and former COO at WTFast. He writes Management Skills Daily to share practical management frameworks that work in the real world.

Recent Posts