Do you want to improve your ecommerce performance? In this article, I will explain what ecommerce ab testing is, what the tools are to implement it, and how it will help you increase the conversions of your online shop.
Low sales, shopping cart abandonment, and poor user navigation are problems that must be analyzed and intelligently solved for an E-commerce to generate a good profit.
Through ecommerce ab testing you can, from time to time, work on your store to develop an effective sales strategy. The key word here is experimenting to find out which solution is the most functional for your business. Let’s see specifically how it works and why you should perform ab testing on your ecommerce.
There are no magic formulas for acquiring customers and improving your business, but with regular commitment and the adoption of appropriate strategies, you can boost your ecommerce performance and achieve significant results. Put improvisation aside and never proceed by trial and error: any action you take must be reasoned, analyzable, and lead to measurable results.
If, on the other hand, you want to learn how to design and implement winning strategies, learn how the online sales market works and learn how to use the main CMSs, sign up for one of our E-commerce courses.
What e-commerce AB testing is for
Through ab testing, also known as split testing, two different versions of the same page are compared at the same time with the aim of testing which of the two achieves more results and thus converts more.
To better understand, let us take an example. Think of page A, called the control, and page B, called the variation. Version A corresponds to the original page, while B is the experimental variation. In both versions, there is a button. The only difference, in our case, is the different colors of the button: in A it is yellow, and in B green. The ab test allows us to understand, depending on the set metrics, which button was able to improve, for example, the CTR or the conversion rate.
The particularity of ab testing is that the results obtained have a solid scientific basis. After all, it is a process borrowed from statistics which, far from any form of improvisation, always starts with a sample, formed by the number of users tested, and a logical hypothesis to take into account:
- From the data gathered through analyzing the initial circumstances you seek to alter;
- Regarding the modification you intend to experiment with;
- Concerning the anticipated effect;
- Pertaining to the metrics used to gauge this effect;
- Within the designated timeframe for achieving the projected outcome.
For the test to be statistically valid, the sample users must be randomly distributed into two groups:
- Control that it will only see the original page.
- Treatment that will instead have access to the experimental version.
The size of the sample to be examined must be established before starting the experiment. The latter, it should be emphasized, affects the quality of the results: the larger the sample, the more reliable the results will be.
To summarise, ecommerce ab testing makes it possible to test every single element of a page, starting with the font, the text, the position of the blocks, and the choice of colors, right down to the landing pages and the entire layout. All this with the aim of incrementally improving the performance of a site. If you are not sure how to structure your landing page or are undecided about which type of text to use to describe a product, with ab tests your own visitors will suggest the best-performing solution.
There are a few basic points to define before starting an ab test on your ecommerce.
- The goal: The first step is to set a concrete and measurable goal and only then can you decide on the element to be tested. Your goal could be to increase the time users spend on a page, reduce the cart abandonment rate, and to increase the number of subscribers to your newsletter. To define this, you could use web data analysis tools, such as Google Analytics, or directly involve your customers through surveys and interviews.
- The metric: having clearly understood the objective and the result you estimate to obtain, you must establish a single metric (e.g. open rate; ROI, CTR, conversion rate) with which to measure the effectiveness of the test. For example, your benchmark metric could be the percentage of CTR that the landing page of your online shop, in variant B, is able to generate when compared to the original A. Also remember that in order to perform the test correctly, it is important to focus on the performance of the individual parameter whose improvement you wish to test and not on the overall performance.
- Target: as seen above, before running the experiment, you must identify the sample to be administered. Depending on the target, the sample can easily consist of the entire public with access to your site or, for example, only registered users. The ab test, in its most classic form, requires that 50 percent of the users, belonging to the control group, interact with the original version and the other 50 percent, consisting of the treatment group, with the B variant.
Another important step is to consult previously performed experiments so as not to forget what you have tested, in which period, and with what results. In order to have easy access to the tests already performed, it is a good idea to archive them. Possessing an archive of performed experiments will save you resources and energy, and provide you with useful indications on what to test in the future.
Types of ab testing for your e-commerce
AB/n testing
There may come a point where conducting a conventional AB test proves inadequate for your experimental objectives. AB/n tests enable simultaneous testing of multiple variants alongside the control page, facilitating comparisons. The sample distribution, in turn, adjusts based on the number of variants.
For instance, when presented with a control page and three variants, each group comprising 25% of users, 25% will view the control page, another 25% will see the first variant, an additional 25% will be exposed to the second variant, and the remaining 25% will encounter the third variant.
Multivariate testing
Multivariate is more complex and time-consuming to perform. It is a type of experiment in which several elements within the same page are tested simultaneously, identifying which combination is most effective. So, in this case, it is the elements on a page and the way they are combined that vary.
Why use ecommerce AB testing
Within an ecommerce we find a number of variables that can positively or negatively influence the entire purchasing process. Page layouts, navigation experience, text, call-to-action design, promotions, and pricing strategy are just a few of them. For your e-commerce to be successful, you have to remove the obstacles that prevent your customer from having an optimal shopping experience. With e-commerce ab testing, if done well, you will be able to make significant improvements to your online shop, because it will allow you to work methodically and consistently on every aspect of the store. Through ab testing you will be able to:
- Increase the conversion rate and sales of your online shop;
- Reduce cart abandonment and bounce rate;
- Improve user experience and engagement by reducing friction;
- Increase the value of the customer life cycle;
- Increase the average value of the order;
- Improve return on advertising expenditure;
- Improve the loading speed of e-commerce by positively affecting its ranking.
Check out the Free E-book and learn
everything you need to know about E-commerce
E-commerce ab testing: how to perform it
E-commerce ab testing can have an important effect on several KPIs (Key Performance Indicators). That is why it is crucial that it is implemented in the right way. Let’s take a look at the steps to follow to set up ab testing effectively.
Information gathering and analysis
Gathering and analyzing information is preliminary to testing. In this phase, you must create your roadmap or testing roadmap that will guide you throughout the testing period, telling you what to test and in what order. Based on the results you obtain, the testing roadmap will have to be cyclically updated.
The first step in creating your roadmap is to study your customers and their behavior. The more you know about your audience and how they interact with your site, the easier it will be to identify those elements that, if varied, could lead to positive results in terms of performance. To gather information about your audience, you can rely on some useful research tools.
- Google Analytics: use it to calculate page bounce rate, and CTR (Click Through Rate), learn about traffic sources, user dwell time on different pages, and the path of visitors landing on your site.
- Heatmaps by HotJar: this allows you to graphically visualize how users interact with your e-commerce using heatmaps. Through heatmaps, you can see where users click, and where they get stuck, so you can understand where to intervene to improve web usability and increase conversions.
- Surveys and feedback: submit your users to surveys and collect information about their shopping experience. Ask them how it was, what they would like to see improved, and what needs to prompt them to buy from you. Investigate the individual steps in the sales process and identify weak points that need to be addressed.
Only an in-depth analysis of your online store will be able to tell you which aspects you actually need to work on, and thus what you need to examine.
Do you like collecting, measuring, and analyzing data?
Test your aptitude and find out with a simple test!
Formulation of the Hypothesis
Now that you have gathered all the necessary information, it is time to turn it into a hypothesis. We can define it as a proposed solution to a problem. In our case, it always stems from a research path; in fact, it must be constructed from data collection and analysis and be consistent with business objectives.
The same must:
- Be measurable;
- Propose the most effective solution to a conversion problem;
- Provide new market data and, consequently, food for thought.
So, to maximize your chances of success in the test, make sure you have a solid hypothesis that can answer the following questions:
- What are you going to test?
- What is the result you wish to achieve?
Establishing a hypothesis also makes it possible to determine the size of the groups to be subjected to the control and treatment versions.
Prioritization
Starting with the data collection, create a list, not too long, of test ideas. Use the list to establish priorities. This will help you to decide what to test and establish the order of tests to be performed in succession. You can also use a simple Excel sheet to create your list.
When assigning priorities, take into account:
- The kind of impact the test will have on the business if successful;
- Of the concrete possibility of carrying out the test, and thus its ease of implementation in terms of time, cost, profit, and organization;
- Of the ability to succeed, i.e. how confident you are that the test will be successful.
Each of these conditions should be given a score on a scale of 1 to 10. By averaging them, you will know which test to start with. To simplify, the first test to be performed is the one with the best cost/benefit ratio.
We have now defined the priorities and the order in which the tests are to be carried out. All that remains now is to choose the software and configure the test for your ecommerce. If you are still unfamiliar, the advice is to start by testing small changes so that you become familiar with the tools and learn how to read the results properly in order to measure them correctly.
Analysis of results
When analyzing the results, do not just check whether the test was successful or not. Reaching 95 percent statistical significance is an absolutely necessary but not sufficient condition for the test to be considered valid in all respects. The reliability of a test depends on the following conditions:
- The result must reach a statistical significance of 95 percent. This is provided automatically by the ab testing tools;
- The duration of the test must have been long enough not to produce false data. As a rule, it should last three to four weeks;
- The result obtained must prove to be statistically significant, otherwise, it will be necessary to start again, choose a new element to vary, and repeat the test. There must not have been any major fluctuations in the graph in the last time frame of the experiment.
Remember that if the experiment fails as a whole, this does not mean that your variant could not have produced positive results. When collecting the data generated by your experiment, it is important to segment the user base and see if on specific segments your hypothesis is valid.
Incorporate A/B testing methods to enhance your Inbound Marketing.
Download the ebook now!
Test duration
Ideally, a test should last until statistically significant data are obtained. As a rule, it is suggested to run the experiment for at least two business cycles.
However, the duration of the ab test should not exceed three to four weeks, nor should it be excessively short. In the latter case, the risk is to come to too hasty conclusions. Performing a test for too long, on the other hand, will make our results erratic and unreliable because, faced with too long a time span, many of the starting conditions taken into consideration may have changed.
When to avoid the experiment
ab testing is not always the most suitable solution for e-commerce owners. It is inadvisable to perform a test if:
- Traffic to your site is low. For an ab test to be reliable, you need to have a good volume of traffic so that you can achieve statistically significant results within three to four weeks, otherwise, there is a risk of polluting the data. For instance, within too long a period of time, your users could delete cookies and be recognized by the test as new visitors, thus distorting the results;
- Special events are taking place that may affect the normal flow of traffic to your site. To be reliable, the test must be performed under standard conditions.
In these cases, it is better to directly adopt other analysis tools such as interviews and user surveys.
E-commerce ab testing: which tools to use
What is the best tool to perform e-commerce ab testing? There is no absolute answer to this question. All the software we are going to look at will allow you to automatically calculate the conversion rates of the variants and determine which version was statistically the most effective. In essence, the choice of software depends on your technical knowledge and your investment possibilities.
Let us explore together which are the main tools with which to set up an ecommerce ab testing:
Can you call yourself an E-commerce Specialist? Answer the questions to find out!
Optimizely
Optimizely is one of the best-known applications for performing ab and multivariate tests. It has a visual editor that allows changes to be made on the pages to be tested without necessarily having to access the source code. It is therefore also suitable for those without specific technical skills such as knowledge of HTML or CSS. It automatically generates the variant to be tested. You can try it free of charge for 30 days and then proceed to purchase a paid plan. Optimized integrates seamlessly with tools such as Google Ads and Analytics and is compatible with various e-commerce platforms. To install it, just copy and paste a simple snippet between the <head> </head> tags of your site.
Create your first eCommerce A/B testing with Optimizely
VWO (Visual Website Optimiser)
VWO’s features are very similar to those of Optimizely: the software also allows you to set up ab and multivariate tests without having to get your hands on the source code. The software can be tested free of charge for 30 days, after which you will have to upgrade the service and choose the plan that best suits your needs. Once registered, before using it, copy the smartcard from the settings and paste it between the <head> </head> tags of your site. VWO integrates with various web analysis tools and is compatible with several E-commerce CMSs such as Magento and Bigecommerce.
Create your first eCommerce A/B testing with VWO
Google Optimise
It is a completely free tool devised by Google that uses a visual editor to create the variant without having to rewrite the code. It is designed to interact with Analytics, Google ADS, Firebase, Google BigQuery, and AMP pages. If you are not looking for advanced features, it is the ideal tool to start with.
Create your first eCommerce A/B testing with Google Optimise
Examples of ecommerce ab testing
We have seen how crucial it is for ecommerce owners to perform this type of testing on a regular basis. Below you will find some interesting examples of e-commerce ab testing to draw on.
FAB: improving the user-customer experience
Fab is an online retail community where members can buy and sell various items: clothing, accessories, home and collectibles, and so on. Some time ago, it carried out ab testing to understand how to improve the customer shopping experience.
Hypothesis: the study of the site’s user behavior revealed that community members used to add products to their shopping cart directly from the catalog page. Fab, therefore, thought of performing ab testing to see if making the ‘Add to Cart’ button clearer would increase the number of people adding items to their cart.
Result: enlarging the size of the button and improving the text of the Call To Action (variation 1) led to a 49% increase in additions to the shopping cart compared to the original version, while the second variation led to a 15% increase.
Considerations: The test demonstrates that a clear, direct, and intuitive Call to Action, capable of smoothly guiding the user through the purchase process, can significantly improve the user customer experience. [optimisely.com].
NuFace: incentivizing users to buy
NuFace is a company specializing in the sale of anti-aging products. Although their e-commerce was generating good traffic, users were holding back from making purchases. Hence the need to conduct an ab test to choose which strategy to adopt to incentivize users to purchase.
Hypothesis: in order to persuade users to place their orders, they tried to offer them free shipping for orders above 75$.
Result: the addition of the free shipping service led to a 90% increase in orders and the average order value (AOV) increased by 7.32%.
Considerations: The ab test performed highlighted how offering rewards or incentives to one’s customers significantly increases the chances of sales. [vwo.com].
Spreadshirt: redesigning the layout
Spreadshirt is an e-commerce of customized t-shirts and accessories that allows illustrators, graphic designers, and artists, in general, to open their own online shop where they can sell t-shirts, sweatshirts, hats, bags, and more with prints of their work on them. In 2013, the site decided to redesign its layout.
Hypothesis: as sellers are the driving force behind Spreadshirt, we start experimenting with them. The conversion goal to be achieved is to turn visitors into sellers. E-commerce thus started testing the homepage, in particular analyzing the section inviting the visitor to sell in an attempt to improve the call to action.
Result: the adoption of simple, clear, and immediate graphics led to a 606% increase in conversions and purchases.
Considerations: The original layout included more calls to action, had a more complex appearance and was rich in text. By eliminating the excess elements, Spreadshirt was able to focus on reinforcing the main call to action, ‘start selling’, and the value proposition, resulting in a significant improvement in conversions. The experiment confirmed that simplifying high-impact areas can increase the number of conversions and the degree of user engagement. [optimisely.com].
E-commerce ab testing: mistakes to avoid
To sum up, what are the main mistakes not to make when performing ab testing on your e-commerce?
- Perform several tests at the same time. Testing too many elements at the same time may prevent you from realizing which change actually led to an increase in conversions. Proceeding gradually may prove to be the right choice for incremental results. Better to perform one test at a time.
- Show the versions to be compared in two different periods. It is essential that the sample, divided into groups, interacts with the variants in the same time frame.
- End the experiment too early or make it last longer than necessary. In the latter case, there is a risk of polluting the sample.
Are you planning to set up an e-commerce, but don’t want to make any mistakes? Book our one-to-one strategic coaching to make the right choice.
