The Dirty Little Secret Nobody Tells You About Website Split Testing...
© Copyright 2005 Eric Graham
Iíve got a confession to make. This may come as a surprise to you (after all, Iím a testing fanatic)... But, testing doesnít improve website conversion every time.
Nearly all "online marketing gurus" promote the benefits of conducting split tests and multi-variable tests on your website. But, few (if any) tell you the WHOLE truth about how to test, what to test and what results to realistically expect from your testing.
The consequences? Many online markers become confused by and disappointed with their first fledgling attempts at conducting testing campaigns on their websites.
Because their results often donít live up to the "hype", they become discourage and simply assume that "testing doesnít work" or "itís not worth the effort".
I hear this from my clients all the time. They will tell me, "oh... I tried testing on my site and it didnít help."
Well... in an effort to clear up a few of the misconceptions about testing and help folks overcome some of the unrealistic expectations that simply set them up for disappointment, Iím about to tell you the "dirty little secret" that the gurus usually fail to mention.
Every variable you test will NOT automatically improve sales. In fact MOST of the variables you test will either reduce response or make no measurable difference!
In fact, after conducting over 6,000 split and multivariable tests over the past six years Iíve generated some pretty good data on what you can expect to achieve in a typical testing campaign.
In most multivariable tests only 1/4 to 1/3 of your variables will boost response. Another 1/4 to 1/3 of your test variables will actually reduce response. And 1/3 to 1/2 of your variables wonít make a measurable difference.
(Well... thatís not entirely true. If you run any test long enough a winner will usually emerge, but it is not worth the trouble to let a campaign run for months on end just for a 1/10th of a percent improvement in conversion.)
Of course, the more experience you gain in testing, the better your odds of finding a variable that will be a winner... but you still have no guarantees.
Often variables that you are sure will boost sales, wind up having no effect or worse, decrease response.
Iíve seen tests where a 1 year guarantee was tested against NO guarantee and the no guarantee version won. Iíve conducted price tests in which we tested $17 vs. $97 and $97 converted better! (Not just higher visitor value, but actually higher conversion!)
So it really is impossible to know in advance what version or variable is going to win. Experience and prior testing data can help you make an educated guess, but you still have to test it and see.
I view selecting test variables, kind of like picking stocks. You can do all of your homework, use a variable that has boosted response on every other site you tested and still not have a winner.
And the opposite can happen. You can test a variable that you are almost positive will be a dog, (or even make an error when setting up your variables) and it can boost response. Iíve often seen headlines that I hated or graphic images that were down right UGLY convert better than the more appealing versions.
So just like picking a stock, past performance doesnít guarantee future results! Every single variable you test will not be a winner. In fact when it comes to the stock market, if you can pick a winning trade just 51% of the time, youíll become rich.
The same principle applies to test variable selection. If youíre able to consistently pick winning variables 50% or the time or more, youíre doing an outstanding job! (Or youíre simply not running enough tests...)
Now, am I telling you all of this to discourage you from testing? ABSOLUTELY NOT!
Like I said at the beginning of the article, Iím a testing fanatic.
Iíve seen small businesses transformed into "big" businesses almost overnight via proper testing.
Iíve seen a laid off, single father go from struggling to pay the bills selling an ebook to making a solid six figure income by only tweaking and testing 4 items on his sales letter.
Done right (and with proper expectations) testing can indeed change your life and transform your business.
I just want to make sure that you go into the process with the right, long term mindset and that you donít become discouraged and quit if your first attempts at testing fail to yield earth shattering improvements in response.
The good news for testing is, because you can flush the variables that donít boost response and only keep the "winners", even if you only pick a winner 10% of the time youíll still make huge gains in conversion and response over the long haul.
A veteran of over 6,000 online tests, Eric Graham is the secret weapon top online marketers turn to when theyíre having conversion rate trouble. To learn how Eric can help boost YOUR conversion rates visit: www.web-site-evaluations.com