Less information leads to better decisions

In digital marketing, there’s nothing that we can’t measure. You can adjust Google Analytics to show hundreds of data points. Facebook provides you with hundreds of metrics. We measure so much that it’s almost impossible to create a bad report. There’s always some metric that’s up.

But our long term goal is to get to the best possible decision that leads to the best possible outcome. That means that we need correct information that we can digest, to see the world as it is, and show us how it could be.

Just the basics

I’ve long been a fan of singling out just a few critical metrics at core points in the process to judge performance. And now that I’m working client-side, I’m doubling down on that stance. As an advertiser with something to sell, I’m not interested in pages per sessions or what the bounce rate is. I want to know how many people:

  1. Came to our site,
  2. Added a product to their cart,
  3. Started the checkout,
  4. And made a purchase

I can run my side of the business fantastically well by focusing on just those metrics. I need scale to justify our advertising investments, and enough of those people need to think our products are worth buying. Obscuring these core metrics with additional ones doesn’t improve my decisions.

Listening to an interview with Adam Robinson brought that into focus. Adam mentions a piece of research by Paul Slovic on how more information influences accuracy and confidence.

Misplaced confidence

In the study, Paul asked horse race handicappers to predict the outcome of a race. The first ten races he would give them any five pieces of information that they wanted. The next he gave them ten, then twenty and finally forty.

It turns out that using just five pieces of information leads to an accuracy of about 17%. That is about 70% better than picking a horse at random. Having more info didn’t increase their accuracy.

Paul also asked how confident the handicappers were about their predictions. That confidence shot up from 19% with just five pieces to 34% with forty pieces. So having less information available leads to almost the same outcome, but with a far better understanding that the result might be wrong.

Confirmation bias is to blame for that mistaken increase in confidence. The handicappers made their assessment with just the first five pieces, and then used the extra pieces to confirm that they made the right choice.

Less information, easier decisions

Confirmation bias leads to more difficult conversations, because not only has someone seen what they needed, they’ve seen more to convince them that they’re right. Changing someones mind will only be harder now.

Simplifying what we measure and what we report leads to far better results in the long term. We only feed our confidence by measuring a thousand data points and investing in black boxes.

Returning to just the basics means we get correct information that we can digest, so that we see the world as it is.

Leave a Reply

Your email address will not be published. Required fields are marked *