Case Study : 48 hours to review a checkout

By Ross A Hall on December 5th, 2016
Case Study : 48 hours to review a checkout

Sometimes you know something is a good idea, but it has to be justified. I was asked to review the checkout flow of a business simply to create a business case for change. A new vendor was already selected, contracts were ready to be signed and then someone in authority asked for an independent assessment.

I enter from stage left.

It was a simple brief: review the as-is checkout flow and provide an honest, no-holds barred assessment of how effective (or otherwise) it was. I was constrained by not having access to real customers or research and a strict time limit. Like a bad action film I had 48 hours maximum to come up with my findings and they wanted some numbers.

To protect client confidentiality I have ommitted some details and data.

Step 0: This is what I need.

Before I started working I made sure my sponsor knew what I needed: which came down to some financial information and analytics data. I also made sure one of their MI teams was available to me to grab data quickly where I spotted gaps.

(I also asked for any design documentation they had, only this wasn’t available.)

Step 1: The Teardown.

Acting as a consumer I bought a couple of things. As I moved through the site I recorded my commentary, capturing particular things I wanted to go back to or reactions to different features.

Once I completed the purchase I went back, repeating the process and capturing screenshots of every page. The screenshots went into a Keynote file with bullet point notes for me to refer back to.

Step 2: The Happy Path.

Going from checkout to “thanks for your order” is a process. The first step is to get that process down on paper, mapping out the flow, screens and behaviours.

From my screenshots and notes I captured the flow in Sketch, using abstract wireframes to represent the flow. In more detailed reviews these wireframes make it easier to see data capture, buttons and other triggers than in screenshot images.

UX Checkout Review

Step 3: The Sad Paths.

If things don’t go according to plan then the user follows sad paths. These went in next, showing where the experience was calling out to different pages or flows. In my notation I call these out using red dotted lines for clarity.

UX Flow

Step 4: The Analytics.

Now I knew what the process looked like I could start to dig deeper into it using data. I constructed a funnel, showing drop-off rates at each step in the happy path and the impact of sad paths. I started to form hypotheses that I could test.

A couple of the key ones included why the abandonment rate on the payment page was exceptionally high, the lack of international sales and why so few account holders logged in.

Step 5: Testing.

Each hypothesis was explored by working back through the screenshots and notes and on-site testing. In a larger exercise I’d prefer to let real-customers loose on a site, but in this instance it was all down to careful review, analysis and in a couple of cases digging into code.

Pretty quickly it became clear there were some significant issues on the site, including non-secure items breaking the padlock on the payments page, additional costs for international shipping not being disclosed to the customer until they entered their credit card details (and even then the price just changed) and a lack of signposting to account login.

Pause: Updating Sketch

Throughout this project I was using Sketch to capture flows, data from analytics and layer comments and findings into their context. I paused for a while just to bring this all up to date and create a single large artboard with all of this information on.

I’ve found keeping everything in one place like this is incredibly useful and powerful. It might never see the light of day as far as the client is concerned, but for me it provides immediate access to information I need and often I can pull patterns and interconnected issues out and high-light them quickly.

Taking a few minutes out to consolidate everything also gives the mind a chance to reflect and review. Sometimes unusual insights can pop out.

Step 6: The Business Case.

In normal circumstances I would produce a business case that validates (or otherwise) a course of action or a recommendation. For this client I produced projected budgets and return on investment figures that could be used to guide decision making. Remember I was flying blind with no knowledge of who the checkout vendor might be or the solution they would offer.

It was a simple spreadsheet that high-lighted the impact of each issue I identified within the booking flow, then assigned a “best / worst / most likely” modifer to the abandonment rate on the page where it appeared. Monthly sales could be plugged in at one end and then the spreadsheet chained backward to work out what the new potential sales could be as the vendor’s solution checked off each option. It wasn’t perfect, but was good enough to guide some decision making.

Final Step: Delivering the News.

The last thing to do was pull together a quick report from one of my templates and have a chat with the client about it over the phone. In less than 10 slides they had a view of their current checkout process, data demonstrating pain points and areas they should address. The Excel tool gave them an extra edge in being able to quickly assess whether the vendor was worth pursuing.

A couple of days later I got an email to say they’d signed contracts and were going ahead as my analysis showed a compelling business case.

Reflecting Back.

The temptation is to look back at projects like this and say, “I could have done more.” With more time I could have commissioned some user research, or dived deeper into some of the layout issues, calls to action and so on. Perhaps I could have even designed a better checkout experience.

But that wasn’t what the client asked for, nor what they needed. What they needed was a quick review to validate a decision they’d already made. If I’d gone the extra mile the chances were I’d have muddied the waters, confused the client and delivered something they didn’t value.

My Artboard

I like to use an Artboard in Sketch to keep all my notes, comments and diagrams together. This is a redacted version of the one created for this project.

UX review artboard

Share this page:
My name is Ross Hall.
I design effective, profitable experiences.
Read more about the work I do, or follow me on social media:
Subscribe for weekly updates direct to your inbox...
Let's talk about your project...

Call me on +44 (0)20 3287 0071

or use the form below and I'll get back to you by Email...

Who are you?
What's your email address?
Which company do you work for?
How can I help you?
Your privacy. You're not being added to a mailing list and your information will only be used to reply to your message.