Don’t pick the numbers to prove your point

By Ross A Hall on December 22nd, 2016
Don’t pick the numbers to prove your point

During a debate over an approach to a particular customer journey two camps developed. On one side an accountant argued his approach was the most likely to win new customers, on the other a UX specialist promoted an almost diametrically opposed view. Neither was supported by robust data, so it was agreed that they would be A/B tested to determine which would work best. Once there was clear evidence in place to support one hypothesis over another that would take precedent.

At least, that’s what everyone thought.

It was an overheard conversation that made me aware all was not as it seemed. The accountant was briefing his junior and asked him to reduce the conversion rate in the investment case because of the decision to use the “other” approach. This was never discussed and never captured. When challenged why conversion rates had fallen it was simply due to “an adjustment”.

What wasn’t clear to me at the time was whether this “adjustment” was a wilful act to deceive or an accidental consequence of confirmation bias. Was his objective to charge in as the knight in shining armour and present an alternative and more effective approach, or simply him applying subconscious bias to a model that was loaded with assumptions?

We’re all guilty of this bias from time to time. We assume our view is the right one (or at least the best) and when challenged see the competitor as inferior. Its faults are writ large, while the problems with our view are seen as trivial or surmountable. If left unchallenged our view dominates our thinking and any opportunity to high light the failures of the other are amplified, exploded and clearly documented.

In my role designing customer journeys and target operating models I have to remain open minded. If my client comes to be with an alternative way of solving the same problem I have to look at it objectively. That can be difficult, although I hope it is a skill I have cultivated over time. Being able to step back, disconnect myself from the hard work I’ve put into a particular design, and revisit everything from a purely objective viewpoint takes time and patience.

For those who design I implore you to do the same. Apply the same rigorous attack to your own work that you might apply to someone else’s. Challenge yourself at every turn and question whether your defence is based in hard data and experience or simply a desire to prove you are right.

As to those who propose a design that is not accepted, or which is modified by others, have the good grace to accept the decision. Don’t try and undermine it through manipulation (whether conscious or otherwise) and instead focus on making the decision that’s been made work. Of course, if you believe you are right so strongly that you can’t support the decision you always have the option to walk away. And yes, I have taken this difficult step.

Whatever the outcome remember that objectivity should always win through in the end.

Share this page:
My name is Ross Hall.
I design effective, profitable experiences.
Read more about the work I do, or follow me on social media:
Subscribe for weekly updates direct to your inbox...
Let's talk about your project...

Call me on +44 (0)20 3287 0071

or use the form below and I'll get back to you by Email...

Who are you?
What's your email address?
Which company do you work for?
How can I help you?
Your privacy. You're not being added to a mailing list and your information will only be used to reply to your message.