During his “2021 Analytics Unleashed” keynote address, the President of Hubbard Decision Research, Doug Hubbard presented an introduction to why nothing is immeasurable. At first, Doug listed many commonly stated reasons people give for not using measurements and quantitative methods. One of them being “We don’t have sufficient data.”
He argued that such a statement is actually a very specific mathematical claim. “When someone says they don’t have sufficient data to measure something, did they actually do the math? They are probably winging it” emphasized Doug. He further added that the implied (and unjustified) conclusion from these objections is: “Therefore, we are better off relying on our experience.” However, it turns out that your experience is still just a model. You are modeling it anyway. The question is never whether to model. It is: “Which modeling method is less wrong?”
Doug followed up by presenting measurement misconceptions using his 3-part method. He calls it the “Dot C.O.M.” approach, which consists of Concept, Object, and Method.
Watch Analytics Unleashed on demand
In the “Concept of Measurement” part of the presentation, he points out that the definition of measurement itself is misunderstood. On top of that, it turns out that people can be trained to be pretty good at subjectively assessing probabilities. Doug also shared a story of how they trained over 2,000 people in the last 22 years to be very good at subjectively assessing probabilities.
In the second part of his “Dot C.O.M.” presentation called “Object of Measurement”, he starts off by highlighting that in most cases the thing being measured is simply not well defined, to begin with. If you ever wondered “How do I measure teamwork?”, “How do I measure strategic flexibility?”, or “How do I measure innovation?”, this section of the keynote is for you. It’s where you will learn why when people try to measure such questions, they always run into a wall.
The “Method of Measurement” part is where Doug mentioned that how statistical interference works is misunderstood. “No matter what you are measuring, assume the following until you prove otherwise: you have more data than you think, you need less data than you think, and there’s a good chance it’s been measured before,” says Doug. He finishes this section of his presentation with The Initial Information Rule which states that “If you know almost nothing, almost anything will tell you something.”
Lastly, Doug closes his 2021 Analytics Unleashed keynote address by presenting a process they have put together called “Applied Information Economics” (AIE). As a process for measurement and decision making, AIE quantifies and then optimizes decisions by focusing onmeasurements where they matter most.
Doug summarized by asking “What is your single most important measurement?”
“It is the performance of your measurement instruments for decision making,” he answered.
If you missed Analytics Unleashed, don’t worry! It’s available for on-demand viewing.
