Is that “one number to measure” really an insight or just an exercise in reductio ad absurdum?
Last week I was speaking to a group about developing different indicators that measure a company’s performance. I brought up a spreadsheet with almost 400 Indicators (Key Performance Indicators KPIs, Key Result Indicators, etc.). As my mind wandered, as it often does while I’m presenting, I thought about how much easier this would be if there was only one Holy Grail KPI we could all use. Then I figured maybe the answer was 42, like in the Hitchhikers Guide to the Galaxy. But that would be just too simple wouldn’t it.
Let’s consider how it works in real life – more or less.
Imagine the head of marketing at a very, very large fast food chain was responsible for reporting on the effectiveness of corporate-sponsored global sales promotions. Let’s call him Bill.
(Did you bring a towel?)
Every month Bill stood before the executive committee and broke down the previous month’s “pillar promo.” He was sure his report mattered. A lot. It had to. After all, big money was spent on these promotions — television support alone often ran into the tens of millions of dollars.
Early in his tenure, Bill religiously prepared 50-slide PowerPoints, dense hand-outs (with tabs!), and reams of supporting detail. He brought 23-year-old quant jocks to sit along the wall and look serious. There were charts and graphs. He had sliced the data by geography and day-part. He knew the impact of regional economic conditions and factored in the weather. He interviewed store owners. He did co-variant analyses. He knew the value of r-squared.
He was allotted 30 minutes, but he was often asked to do it in 10 “because they were running a little behind.”
As months and years went by, Bill started distilling the entire analysis into 10 KPIs and commentary. Then five and no commentary. And that’s when it hit him: what the exec team really wanted was one number. A single solitary number that told them the answer… or made them think they knew the answer. They didn’t care about nuance, about what the information might be whispering to them if only they listened.
After a while, Bill’s reports went like this:
“Whattya got, Bill?”
“The Burger BOGO did a 3.6.”
“Thanks, Bill.”
They knew it was a composite score out of 5. They also knew that Bill’s numbers tracked loosely with revenue and with the angry or congratulatory calls the CEO got from the largest franchisees. But that’s all they knew, and it’s all they cared to know.
There are several morals to this story. Certainly one is: don’t bleach all the goodness out of your performance measurements in search of a “power coefficient” that tells the whole story. Our rich and varied world resists such attempts. Simplify, but don’t over-simplify. You’ll probably need 5 to 10 key measures to drive real value from your business.
I think Bill’s experience is being replayed right now in the world of social media measurement. There’s a school of thought that says the Holy Grail is “engagement”.
Very trendy, but it’s not “42”.
One reason everyone has latched onto engagement seems to be its relationship to the popular “one number to measure” technique used to estimate the future value of a brand. Called the Net Promoter Score, or NPS, it was introduced by Fred Reichheld in 2003 and promoted through the consulting firm Bain & Co.
Simply put, the NPS is the number of brand users who say that they will mention a brand to their friends. That number is then netted against the number of those who say they won’t. If the “promoters” outscore the “detractors” on the NPS calculation, the score is positive. You get the picture.
I encourage you to learn more about NPS, but here are some of the much-debated pros and cons:
- Popular with executives (Bill would say “Duh!”)
- Easy to administer
- Tracks with revenue
- No better than other conventional loyalty-related questions
- Significant information is lost and variability increases (Again, just ask Bill)
Marketers have come to believe that more engagement drives up a brand’s NPS score. What could be simpler?
I’m wary of engagement because it’s fruit from the poison NPS tree. Others, like Don Schultz of Northwestern University, suggest that social media measurement mavens refuse to admit that engagement is just traditional sales promotion with a “Like” button in the vicinity.
Today, we see the frenzy of professional and academic communication on engagement. The problem, of course, is that no one has clearly defined engagement much beyond “the process of consumers recommending a brand to others.”
Engagement, the way that it is seemingly being defined and practiced today by marketers and supported by academic studies, often seems to be nothing more than a reinvention of one of the oldest tools in the marketing arsenal: sales promotion.
From what I see, it seems that the new social media gurus and advocates have lost sight of what they believe that they are developing. They call the interested consumer behavior “engagement,” but, in truth, it is simply providing various new electronic sales promotion devices through which either offers can be distributed, or responses generated. Engagement should mean that there is some long-term, reciprocal value being developed between the buyer and seller.
Engagement is not dependent on some type of reward that is offered and, if a response is achieved, withdrawn.
Some KPIs are undoubtedly more important than others, but we should resist the temptation to create a single measurement that captures the richness, complexity, and messiness of the world. I’ll be covering more about KPIs in future blog posts.
Until then, as with any metrics we measure, in the words of Douglas Adams “We demand rigidly defined areas of doubt and uncertainty!”?
Ray Major is President and CEO of Halo Business Intelligence. A data scientist, economist and statistician by training, he’s a life-long practitioner in the mysterious arts of data intelligence and analytics, You’ll find his blog here every week.
You can follow Ray on Twitter at @majorbi
