We Ignore Data Because We Are Delusional – by Jason Kilgore

I love data.  I teach college statistics for fun.  Data sheds light on the truth and sometimes that truth is offensive, causing us to become defensive.  Sure, it is always worthwhile to question the data source, veracity, and collection method.  Yet, even when the data is solid, we ignore data we don’t like.  I call it Data Denial Syndrome.  And we are all guilty of it.  As defined by me, DDS is a condition whereby we emotionally ignore what our analytical brain knows to be true.  Here are half a dozen delusions commonly associated with DDS.

1.  Delusion:  “The data doesn’t match what people are telling me.”   Reality:  People are telling you what you want to hear.  Stories people tell are anecdotal and are true only in the singular moment when the event occurred.  Meaningful data is a summary of all the facts, including the facts we subconsciously filter out.

2.  Delusion:  “Statistics often lie.”  Reality:  Good statistics mirror the larger population.  When appropriately collected, the samples answer a very specific question.  Sometimes we try to use specific results to answer a question for which that data set was never intended to answer.  Data doesn’t lie, but sometimes we ask it to answer the wrong question.

3.  Delusion:  “We’re still better than half the companies out there.”   Reality:  There are 120 teams in NCAA Division 1 football.  It is unlikely that you know which team was ranked 59th, just two spots above the bottom half.  It was Wake Forrest and they had a losing record.  And, they are perennially irrelevant to all but a handful of alumni.

4.  Delusion: “The data looks bad, but we are the exception.”  Reality:  No you are not. In fact, those claiming to be the exception are often in the majority.  The truly exceptional don’t play the “we-are-the-exception” card.  They simply accept the data, act decisively, and fix their problems.  In so doing, they become exceptional.

5.  Delusion:  “This data is an anomaly.”   Reality:  Ted Williams had a 0.344 lifetime batting average.   Only once in nineteen seasons did his yearly batting average drop below 0.300.  More often, his average exceeded 0.400.  If your “bad anomalies” occur more frequently than your “good anomalies,” your “bad anomalies” are, in reality, a trend.

6.  [My favorite] Delusion: “The sample size is not valid.”  Reality: Sample size correlates to certainty, not validity.  Without boring you with the details, increasing the sample size of a data set allows us to see the data with more precision.  It does not render the information invalid.  If I were to randomly survey five of your customers and all five say your company sucks, it is not fair to say that their responses are invalid.  We can say with certainty that those five people may or may not represent 100% of your customers.  Just because the sample size is five does not make those responses any more or less valid or any less actionable.

Here’s the simple truth:  Positive data is not actionable, but negative data highlights opportunity.  Data that points to our flaws gives us the opportunity to improve.  Therefore, time spent trying to excuse, invalidate, or justify unflattering data is time wasted.  Use data to understand weaknesses and available resources to improve processes.

Advertisements
Leave a comment

1 Comment

  1. In order to use data effectively you need to understand what it is telling you (and what it isn’t). That is what the first 2 points get at. Often leaders don’t understand data well enough. And instead of figuring it out they just ignore data.

    Data can’t lie. People lie, not data. Data can be used to mislead you, when you are ignorant. Or you can mislead yourself.

    http://management.curiouscatblog.net/2007/08/09/data-cant-lie/

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: