How do we authentically make data-driven decisions in a world with ample data to cherry-pick from?
In a world where startups, investors, marketers, statisticians, executives and HiPPOs (Highest Paid Persons Opinion); all live and die by data metrics, it's imperative that we're looking at the right data. Data is often considered to be impartial and irrefutable; few have the confidence to doubt it, even fewer can constructively argue its source, sample and presentation methodologies. But, data analytics is a complex amalgamation of puzzles and mysteries.
Puzzles vs. Mysteries
(Frank and Ernest by Thaves via gocomics.com)
A New Yorker article Open Secrets by Malcolm Gladwell explains the difference between Puzzles and Mysteries. I'm paraphrasing for relevance to our current context:
Puzzles signify a lack of clarity or information available or visible to us. A puzzle may require an information gathering session or a simple restructuring of information to devise an actionable solution.*
Mysteries require judgments and the assessment of uncertainty, and the hard part is not that we have too little information but that we have too much.*
The KPI Paradox
Doesn't matter your role or rank in the organization, you're accountable for a KPI, or several. Whether you know the exact KPIs or not is a whole other story. We've bastardized KPIs by building dashboards and scorecards, churning and burning through them every quarter. We’ve been writing stories and cherry-picking data to justify our stories, a post hoc ergo propter hoc fallacy.
(Frazz by Jef Mallett via gocomics.com)
Trouble stems when we try to define KPIs and “find data” to solve Mysteries, albeit they are really meant to solve Puzzles. KPIs are leading indicators, and each needs to be tied to a definite action, sort of like the gas light going off on your car.
KPI = gas light on OR off → Action = Stop for gas OR keep going
Business decisions aren't as binary as the level of gas in your car, but they require solving both Puzzles AND Mysteries, and countless KPIs or more data isn't going to get us there. We really need to stop using "more data" as an answer to our decision-making problems. This isn't Hollywood, you don't get to have a script and then find locations, actors, costume et al. to match your narrative.
What we need to do is to differentiate Puzzles from Mysteries; and doing so will lead to effective distinctions between actionable KPIs and colloquial KPIs. The distinction sometimes gets murky, and requires practice and restraint. And oftentimes, a Mystery is only uncovered after solving the precursory Puzzle(s).
Solving Puzzles vs. Deciphering Mysteries
With the ability of tracking every visitor/user action and behaviors, we’re in an entirely new era of passive data collection, an enormous amount of data. Some of this data is invaluable in solving linear Puzzles by answering the “who”, “what”, “where”, “when” and the “hows”; and solving them is key to the survival of our marketing campaigns, product roadmap, and our businesses' survival. But, should we stop there? No! Each time we solve a Puzzle - there’s likely a larger Mystery lurking in the end. These Mysteries are often discovered by asking "What if?", "What then?", "What else?", "What now?."
“Solve Puzzles to Survive, Decipher Mysteries to Scale.”
The expected outcomes of asking those questions is also significantly different between the two.
- Solving a puzzle will lead to actionable insights for effective progress towards your objectives.
- Deciphering a mystery will merely lead to identifying uncontrollable patterns in your users and market behaviors. It requires less data, more foresight and risk tolerance.
Question to ask?
Will solving this (Puzzle or Mystery) lead to actionable insights for me to make changes or will it simply lead me to identifying more of the unknown?
If the answer is Actionable Insights. It's a Puzzle, let's dig into the data and find a way to solve it. If it's More of the Unknown, you have a Mystery on your hands. You'll need to unravel enough of the nesting dolls to reveal the true mystery. And, at some point, maybe at the last atomic level, make decisions and take actions based on gut feeling, experience, and/or optimism.
For it is this unhinged resolve that could lead to groundbreaking insights towards market-gap and revenue stream opportunities. That said, it definitely does not involve more chaotic data collection. We got enough of it already, and more data isn't going to give us an answer.
“Puzzles come to satisfying conclusions. Mysteries often don’t, rather they are the doorway to a journey towards exponential growth.”
In Conclusion
Puzzles are straightforward, questions that have answers, crosswords that can be filled in. Day-to-day business decisions require solving these puzzles, defining your KPIs based on levers you can pull or push to incrementally improve and survive the ever evolving competitive matrix.
On the contrary, Mysteries don’t need revelation; they require prevision, a prophetic foresight based on trends analysis, market forecasting, loony optimism, and plenty of moxie.
Pitch
Here at AKOS., we help decipher mysteries for growth-stage organizations in high-impact industries through Digital Transformations, Platform Modernizations, and Consulting. Give us a shout if you're looking to decode one for your organizations' breakthrough growth.