Every online business should be focused on conversion – it goes without saying. A/B testing (which I’m including multivariate testing in even though they are different) is a great way for businesses to help determine what works and what doesn’t when it comes to converting leads into customers. The recent rise in availability of A/B testing tools is helping many online businesses gain valuable insights they never could before, and to test ideas more accurately than ever. But should it be given the credence it seems to have at the moment?
The thing is, I don’t think A/B testing gives a very complete picture. A lot of people and businesses seem to see it as the be all and end all, if the data says it converts better than that’s what we will go with but I think this is short sighted. A/B testing can tell you what works for short term goals and specific interactions but we need to keep sight of other things like trust and brand perception that are much harder to measure and I’d suggest have a much greater long term impact on conversion.
To help illustrate my point let me give you three examples:
- The first example is changing a button label from “Download our software” to “Get started now”. I’m sure in testing that “Get started now” will be clicked on more often. But is downloading the software what they expect from clicking on this button and how do they feel about what happens?
- Another example is changing a newsletter subscription entry area from an in page form to a pop-up dialogue (also known as light-box). Again I’m sure the pop-up will get more conversions than the standard form on the page but what does it communicate about your brand/personality by pushing it in someones face? I equate pop-ups like this to those pushy sales people on the street who try to get your attention when you are focused on something else.
- A final example is having a check box in a checkout/payment form that asks customers to check the box to NOT receive email newsletters. Again this would no doubt get more subscribers than a straight forward question but what impact does it have when an unwanted email arrives in the customers Inbox?
Don’t get me wrong, I think A/B testing is extremely valuable and I use it when ever possible to help inform the designs I create, I just think it’s important to consider the findings of it in context of other factors – specifically brand personality, trust and long term business goals.
There are a wealth of dark pattern UI tricks out there that convert better than what I’d call the “right” way of doing it but clearly these won’t help in creating a valuable brand that customers want to tell their friends about. And let’s face it, word of mouth is crucial for online businesses.
As well as A/B testing it’s important to do regular user testing. User testing will help build a picture about the more important question you want answered – why. Why are customers drawn to it, why are they more likely to click on it, what are they seeking etc.
Focus on creating real value for customers and on supporting them to do what they want to do in the order they want. Be timely and contextual in the delivery of your messages and offerings. Be true to your brand values and your conversions will follow naturally. A/B testing is a valuable tool in the toolbox but it’s not THE tool. I’m certainly looking forward to when the buzz surrounding it settles.