I have to confess; I skipped the Advanced Experimentation Analysis course. Too technical: coding is not for me. I enrolled in the CRO mini-degree to manage really well tools and statistics and I am also really strong on statistics, but I am not a developer nor do I want to become one.
So… is that bad? Well, actually, I think it shows the quality of the course. The intent with which I enrolled in the course is advanced, but not so advanced as the course: I wanted to stop a bit before coding and getting into advanced experimentation with websites. I the course had stopped there, we could have said that it was not a course for CRO experts, just a beginner’s course. Once again, the CXL institute proved me that they are serious about Conversion Optimization or training in general, for that matter.
It is really in-depth training, it is not just an introduction. Yes, you can understand that by the length of the course – 70+ hours of training – and you also see it by its content. This is a recommendation to those that are really serious about Conversion Optimization – as I am approaching the end of the course, I can now recommend the course But not to everyone. This is not a course for those that want to understand what CRO is and that’s it: you would skip most of the lessons, as they go really deep into all topics related to Conversion Rate Optimization.
Ok, so what have I done this week, apart from skipping courses? I have completed another big part of the course which I actually really enjoyed, although it was also technical, especially the – quite long – course on AB testing.
This is of course the core of the CRO mini-degree and I really liked the approach – technical but not too into useless details, and also strategic, to make you understand how things go down in companies and how to behave and collaborate with other team and management.
The course shows how to conduct an AB test from A to Z, starting from the set-up and ending up with considerations on how to manager AB tests in teams and with companies that have multiple team with different requirements and business needs.
The really interesting part is to see the whole process up-close, so that you get a really good taste of what a professional AB tester does and you understand how important this role is inside a company with many conversions. Indeed, this is another interesting aspect you see inside this course: not all AB tests are done right and actually, there are many rules and errors that may jeopardize the whole optimization process. You learn the minimum conversions you need to have to even start thinking about AB testing and the threshold after which you can start using automation tools to speed up the testing process. Now I understand why AB tests done on Facebook with a past consultant always (and I mean, 100% of the time) gave no results – conversions were too little, or the population that we analyzed was too small. Nobody said it to us and I suspect that with this course I know more than many so-called AB testers that are a bit improvised and that do AB testing because it is “the thing” and everybody else does it.
The course also shows you potential mistakes you can make and the errors you need to take into account to avoid calculating revenues that will not materialize. Very interesting stuff, and very in-depth knowledge that allows me, as I will never become an actual AB tester, to immediately spot if an AB test is done correctly or not.
The course also shows you how to manage AB tests – and that was the most interesting take-home message for me.
You understand that you need to carry out a lot of AB tests as many will fail and then factor in all potential errors to create a business case.
You see also the other side of the coin: management. This is I think a very important aspect to understand, as miscommunication is frequent between technical team and managers, as they see the same goal from two different perspectives and often actually have different goals. Management has to see the bigger picture too and is not too concerned about the result of a single AB test, while a technical team may be hyper-focused on tests and optimizations and will lose sight of the strategic, longer-term picture, which may be dangerous.
In the course you learn how to create a business case for manager, to present the need for new tests and for changes in a website and to present results.
This one of the most interesting part – actually, it was not a “part”, really. The trainer keeps your vision and perspective open during the whole course, to give you an idea of how things work in reality. Once again, you get to peek into the world of the professional and you do not get just a very theoretical course.
Once you finish this part of the mini-degree, you know how to organize a full series of tests, giving them the right priority, to continue optimizing your website or landing page.
You also know what not to do: do not start AB testing if you have 300 conversions, as the tests will not be accurate, the sample is too low. Do not stop after 3 days of testing, as many do, especially when they reach a very exciting result – which does not last long though.
I liked the practical aspect of some suggestions that, in theory, do not respect statistical analysis paradigms, like making many changes and testing them all together if you do not have thousands and thousands of conversions, as your objective in this case is to optimize the page and earn money and not learn exactly which change weighs more than others. That comes later.