Test your tests

Finally, with a month to go and two thirds of the course completed, we get down to testing. Wait, wasn’t this supposed to be all about Conversion Rate Optimization? And wasn’t it supposed to be ALL about testing? Not if you want to do it right. And by right, I mean REALLY right.

When you approach digital marketing and you start following “experts” you run into all sorts of people: great professionals and get-rich-quick-with-me gurus are two categories that immediately come to mind. Unfortunately, the second kind is very persistent at communicating on social media and convinces people who want to make money without effort to part with their money with them thanks to bold statements that makes you fantasize about the new Lamborghini you will buy at the end of the year.

One type of these bold statements revolves around Conversion Rate Optimization. They tell you fantastic stories about how they took a small e-commerce and made sales skyrocket. Percentage growth are always three digits, like “270% growth in two months!”, complete with images of Facebook business managers and bank account statements to prove their incredible wizardry. If you look closely, though, things do not add up even if you don’t know anything about Conversion Rate Optimization: the e-commerce that received such an incredible boost, is still a small e-commerce. The professional claiming to have quadrupled its clientele, earning “six figures” thanks to these CRO gurus still has the same crappy website that looks like it was designed by a drunk cousin – by hand.

Then you meet the guys at CXL and things get real. Very real. They do not teach you anything about testing until this point into the course because, simply, you are not ready. When you start learning about testing with the short course on the most frequent testing mistakes made, you clearly understand that those gurus were faking it, yes, but another, deeper truth comes to the surface: they also knew squat about Conversion Rate Optimization.

They declare to have lifted sales of a small e-commerce with 50 SKUs in one week, when you learn immediately that you need to test for AT LEAST two business cycles and, again, Peep Laja himself would tell you that sometimes this is still too early to quit testing. Moreover, 200-300 SKUs are the bare minimum, and sometimes even those numbers are too small for most of the testing. Then it hits you: these guys are really serious about testing and also about training. They do not want you to think that you can us CRO tactics in every situation and they explain you exactly how to run tests and also… test your tests. It’s only the beginning, but you immediately understand that they do not care about big, bold statements at CXL, they care about accuracy of data and results. So, even if it takes longer, even if it is more complicated, you make sure that your rests are reliable, that your sample represents the population, that the sample size is large enough or that the test you run is long enough to give you reliable data.

So yes, you start by auditing your Google Analytics data: first they teach you to thoroughly set up a GA account, then, for good measure, they throw in another course on the audit, with specific items to be checked.

Then Peep Laja tells you how to run tests and, boy, you need to learn your statistics. You need to learn the numbers that indicate if the test is reliable or not, or if you need more data, more test runs, more samples.

You learn also how to form an hypothesis: “never go into a test for the sake of it” is what you understand from this part of the course. First you form an hypothesis and then you test it: you do not just say “what the heck, let’s see if a blue button works better!”. Tests, when run properly, cost, both in money and effort, and that’s why you start eliminating hypotheses that do not promise great returns. Yes, you need to prioritize and in the course you learn a couple of tricks to score potential CRO activities and how to choose the most effective one.

Now I understand why, with my “facebook consultants” (sorry, I really need to put the words into inverted commas, as I have once again confirmed that many “digital consultants” took too many training shortcuts – with CXL, no shortcuts my dear, you need 74 hours to become a CRO expert) made all sorts of “A/B tests” on small Facebook campaigns, changing small things, like images or titles, and Facebook would always – and I mean 100% of the time – answer: “sorry, there is no clear winner, you do not have enough data”. At that time, I thought “well, isn’t Facebook business manager crap!”. Now I know better. We used to run the test for one week – I can see Peep Laja cringe at the thought – 5 Euro a post per day, and after a week (actually, after 5 days, as we excluded Saturday and Sunday, sorry Peep, I now know you should not do it) they would tell me “mmh, no difference”.

Guess how effective our Facebook campaigns were with those consultants…

Anyway, the start of the third chapter of this course is where you put all the things you learned into practice: you start to think like a real CRO expert.

One suggestion – although I should have probably already said that in another article – for those taking the course: write everything down, as you will need to keep a small guide next to you the first few times you try your own tests, as there are many things to consider and it is not easy. I also see that this clarifies my thoughts, as when I write down this part, full of tests, analyses, statistics, numbers, if something is not clear, I know, as I do not know how to summarize it.

So remember, before testing, you need to learn how to test… and how to tests your tests.

Posted in Blog