When my partner and I were driving through the Luxembourg countryside, we enjoyed the beautiful sights: scenic mountains with hilltop castles and lush forests. As night began to fall, the fog descended. At first, it was beautiful. Then visibility became more and more restricted. Relaxation turned into apprehension as we realized the increasing danger on the road. We could no longer see the countryside. We could barely see the road at all.

The information surrounding complex decisions is like fog. Everything is obscured. Numbers can appear to clear up the fog — but numerical ratings obscure the details behind them — just like the fog’s water droplets keep us from seeing the road.  

Teams often face complex decisions. When they do, we often recommend using decision tables: charts that show options running across the top, and criteria listed on the left. We recommend a set of symbols to go into the intersection between an option and a criterion. Our most frequently asked question about decision tables is an effort to reduce the fog: Can we use numbers to rate the options and calculate a score? Won’t the score help us make a better decision?

Three Reasons Why Good Decisions Can’t Be Scored

Here are three reasons why teams should NOT rely upon numerical ratings to select among multiple options:

Using Numbers Cuts Off Discussion

Inevitably, a team member will often say “Why don’t we add up the numbers and see which one wins?” The result, particularly if there is enough surface agreement or a polite group: a quick, but superficial decision with very little discussion focused on the merits. Sure, there may have been a quick compromise by averaging ratings to come to agreement. But no real discussion, no relationship-building, no insight into each other’s thinking — and a very real danger of falling prey to groupthink.  

How does this play out if there are strong opposing opinions? Often with the numerical ratings already stated upfront, the battle lines are drawn. Curious inquiry into other group members’ reasoning is short-circuited into an attack and defend mentality, however civilized, of justifying one’s own ratings over another. 

Either way, the focus is on the numbers that individual group members are using, and not a healthy group discussion of the reasoning or evaluation that have led individuals to their conclusions. Rather than discussing the alternatives and the reasoning behind the criteria, all of the focus goes to the numerical ratings.      

Using Numerical Ratings Builds a False Sense of Certainty

The very focus on fighting for positions such as concrete numerical ratings can inflate people’s sense of certainty. The more a person argues for a position, the more they convince themselves of the rightness of that position. Numbers seem more objective than good, solid, yet not easily quantifiable reasoning. These two effects combine to produce the false sense that the score reflects objective reality.

Most numerical ratings are only estimates or subjective ratings of unquantifiable criteria, yet using them often leads to greater certainty — because numbers equal objectivity. Some groups may assign values to the relative weight of various attributes, and use multipliers or other formulas to give a more sophisticated numerical rating system. But this greater thoughtfulness still relies on untested assumptions of relative value and ranking. This may lead to an even greater false sense of security.  

The strong certainty from numerical ratings make it more difficult for team members to introduce areas of ambiguity. Yet these areas are often the most crucial in complex decisions — the very reasons why the decisions are difficult enough to merit the group’s time.  

If team members shared their reasoning for favoring particular alternatives, their opinions would be subject to review, discussion and refinement. This would facilitate better reasoning, greater alignment and group learning. When numerical ratings overshadow discussion of reasoning, a false sense of security often leads the team into having comfort with a poor decision-making process.  

Numerical ratings mask poor group decision-making process

Numerical ratings provide a comfortable alternative to the risks group members will need to take for an effective decision-making process. It is easier to assign numbers than it is to take risks such as sharing ambiguity, challenging other’s reasoning, or exposing one’s own reasoning to the team’s criticism.

These types of risks will help the team build relationships and learn from the discussions and the resulting decisions made. By sticking to a superficial discussion of the numbers, the group can avoid these uncomfortable and more demanding discussion topics.  

If the group never gets into sharing of reasons for or against, or for why a particular route is an attractive option, the group does not learn how to handle difficult, ambiguous decisions. If the team uses numbers to avoid discussion on early decisions, they can hide their lack of these team competencies when they can be most easily dealt with, before they reach tougher decisions later in the project.  

Instead, the attack-and-defend mentality can overwhelm the gray-area thinking needed to come up with good options to solve complex problems. The first framing of the conversation can be difficult to redirect.  The team derails their decision-making process by arguing over numerical ratings.   

Why Can’t We Use the Numbers to Get a Fast Decision
and Adjust Later?

Why can’t we use the numbers to get a fast decision, and then adapt and adjust as needed from there? This is a tempting idea, particularly if a group doesn’t have experience of making effective decisions together or only has an experience of participating in — or being an audience of — an attack-and-defend group dynamic. But the numbers hide some of the arguments for and against the alternatives, and distort others — just like the fog on the road.

Some Things are Quantifiable — and Should Be Evaluated
with Numbers

Don’t take this to mean that numbers never count or shouldn’t be factored into group decision-making. Some things like cost, price, time-to-implement can be accurately estimated. Just don’t conflate numerical ratings of subjective criteria with hard numbers. They should not be used as shorthand for evaluating options. Use real-world numbers, and don’t be afraid to challenge or test the assumptions behind those numbers. But use reasons instead of formulas — share your thinking about why you think certain options are better or worse, and don’t rely upon estimated numerical options to substitute for the robust conversation that will emerge.    

What Should You Do Instead?

Instead of assigning numbers to your ratings, use symbols. Symbols are much less likely to be mistaken for certainty, so they foster deeper discussions among team members about the reasoning behind the ratings vs. the scores. At the Rapid Learning Cycles Institute, we recommend a simple set of plus and minus symbols along a five point scale as shown to the right. You could also use stars or some other set of symbols — just nothing that can be converted to a score.