Plane Crashes, Software Failures, and other Human Errors

Hacker News LinkedIn Facebook Twitter by Abby Fichtner

Want to know if your team is effective? Listen to them.

Plane CrashWe can learn a lot about team effectiveness through research that’s been done on teams whose work can mean the difference between life and death. Namely, operating room teams and airline cockpit crews.

The airline industry, which gets such a bad rap, actually has a phenomenal safety record. An airline like United only loses a plane from an accident about once in every four million flights.

Hospitals, on the other hand, which we tend view as safe, are the cause of 44,000 – 98,000 preventable deaths in the US each year. Even at the low end, this tops the death rate from things we actually know to fear like breast cancer (~40,000), car accidents (~40,000) or, of course, airplane fatalities (~120).

So, clearly, the aviation industry is doing better in this respect, and the healthcare industry is now actively working to adopt some of their training, known as Crew Resource Management, or CRM for short.

What is it that they’ve learned? And, can it help us in software? Well, as I like to say, stop me if this sounds familiar…

About 80% of all (airline) crashes are caused by human, rather then technical (mechanical) errors. And, in fact, many of these are the result of errors in communication. Okay, this makes sense for software errors, right? We all know how horrible everyone is at communicating with one another. But, it’s one thing when the risk of miscommunication is frustrated users. If we put ourselves in a situation where failing to communicate could result in “oh my God, we’re all going to die!!” then it seems we’d be able to do just a little better, right?

Let’s take the 1982 Air Florida crash in Washington DC that killed 74 of it’s 79 occupants plus 4 motorists on the 14th Street Bridge that it struck. The first officer tried several times to tell the captain that the plane had a dangerous amount of ice on its wings, enough to cause all of these deaths. And yet, listen to how he reports the problem:

Try #1: “Look how the ice is just hanging on his, ah, back, back there, see that?”

Try #2: “See all those icicles on the back there and everything?”

Try #3: “Boy, this is a, this is a losing battle here on trying to de-ice those things, it [gives] you a false feeling of security, that’s all it does.”

I don’t know about you, but when I picture myself in a situation where my life, and the lives of 78 people around me, is at risk, I’m picturing a very direct and crystal clear warning and if that doesn’t work, screaming, with potentially some jumping up and down and arms waving around. Not “ahhh, see all those icicles back there and everything?”

The captain clearly didn’t take him seriously either until at the end when the first officer finally says, “Larry, we’re going down, Larry,” and the captain responds, “I know it.” Crash. Boom. Bam. 78 dead.

But even more amazingly, what researchers found was that this exact problem was actually common in airline cockpit crews. A famous crash in aviation is the 1990 Columbian Avianca 052 crash that slammed into tennis champion John McEnroe’s father’s estate in New York, killing 73 passengers. The reason? Plane ran out of fuel. Ran out of fuel because it had to go further then expected? No, ran out of fuel while politely waiting for air traffic control at Kennedy airport to give it permission to land. Why didn’t the co-pilot just tell air traffic control they were about to run out of fuel? They were trying to be polite rather than pushy. Peace, love and happiness. Have a flower. 73 dead.

Korean Air Flight 801, almost the same exact situation as Air Florida. In trying to warn the captain of severe weather problems that would eventually lead to the deaths of 228 of the 254 people on board, the first officer says, “Don’t you think it rains more? In this area, here?” and “Captain, the weather radar has helped us a lot”

Captain, the weather radar has helped us a lot?! What are these people doing? They’re hinting at the impending problem, in hopes that the guy who’s a little busy with the whole “flying an airplane” or “trying to bring 99 planes circling Kennedy airport in for a landing” thing is going to catch on, read their mind, and solve the problem for them self. All without losing face. All without getting upset at the hinter for being too pushy or impolite. Wouldn’t want any hurt feelings, now would we?

The official term for this is “mitigated speech” and Malcolm Gladwell provides a fascinating account of how it has effected the airline industry in his book Outliers. He defines it as “any attempt to downplay or sugarcoat the meaning of what is being said.” and explains that “we mitigate when we’re being polite, or when we’re ashamed or embarrassed, or when we’re being deferential to authority.”

Amazingly, this mitigated speech actually explains a major source of airline crashes in the past, and since implementing CRM training to combat it, preventable accidents in aviation have decreased by 50%. And so now the healthcare profession is looking to benefit from it too, because, wow, 44,000 – 98,000 preventable deaths per year!

The crazy common point in both operating rooms and cockpit crews, the thing that makes this all so damning is that research into both fields has shown that errors occur most often when a senior, experienced person is performing. In other words, even in these highly trained professions, it is not just about being skilled or knowledgeable enough – in fact, the more skilled you are, the more it can hurt you!

» In surgery, errors occur most often with experienced surgeons, typically resulting from a breakdown in communications such as a resident failing to effectively communicate vital information (ahh, see all those icicles?

» In commercial airlines, planes are flown with 2 pilots –- the captain and a first officer –- who split the flying duties equally. And yet, crashes occur most often when the captain is flying. Why? “Planes are safer when the least experienced pilot is flying, because it means the second pilot isn’t going to be afraid to speak up.” (Gladwell)

And so the question we have to ask ourselves is – can this also be true in software development? Is being skilled or knowledgeable enough? Or can it sometimes actually hurt us? Do we work in environments where people are encouraged to speak up, to stop the line when they see things going wrong? Pay no attention to the man behind the curtainOr might we have senior team members and managers that are intimidating or difficult to raise issues to? Do we find ourselves hinting to save face in the light of possible problems? Or, perhaps, just remaining silent on issues? (pay no attention to the man behind the curtain

Part of the research behind CRM found that, even among highly skilled professions, human errors are still going to happen. However, we can compensate for this and drastically reduce the consequences of those errors if we have a team that is effectively communicating with one another. A team where its members, regardless of position, are able to speak up and clearly communicate when they notice a problem. And thus, we don’t have to be infallible if we can work with others we can rely on to have our back and catch issues that might miss our attention.

I like this quote from the Nebraska Medical Center, one of the medical centers adopting CRM for their operating rooms:

“The use of CRM creates an atmosphere of mutual responsibility not only for making sure everyone does his or her job, but also for making sure everyone else on the team is informed.”

I’m curious for people’s opinions. Do you think this is an issue in software development? I know us developers can be almost obnoxiously outspoken when we see problems in our areas of expertise… but what about when we see problems in the areas we may not consider ourselves “the expert” at?