When setting up a new message, go to the Design tab of the Message Builder and click the Create variant test button on the right-hand side. This will duplicate your original message so that you can make changes to the second variant. Your change can be as simple as a different headline or image, or it could be a completely new message.

Once you set your message live, half your website visitors will be shown the original message (variant A) and half will see the modified message (variant B). Triptease then tracks how each website visitor interacts with the two variants of the message so that you can see if your changes affect customer behaviour. 

Once you’re happy that one message is outperforming the other (for example when the click-through rate for one variant is significantly better) you can choose to mark that message as the ‘winner’. This will mean that the variant test is complete and all your website visitors will only see the winning variant of the message from now on.
​ 

The variant testing feature allows you to test two variants of the same message and collect data on how they performed. This means you can experiment and test different ideas and theories about what might make your customers book. 

Using variant testing consistently - rather than just once in a while to answer a specific question or settle a disagreement - allows you to continually improve your messaging over time, and make decisions based on data rather than opinion. 

You can check the performance of your variant test in the Message Manager. As time goes by, you’ll be able to see which variant is performing better in the metric you are looking to drive. This will probably change depending on what exactly you are testing. For example, if you were testing colours or images, you’re probably going to be more interested in the effect on your click-through rate. If you’re testing two completely different promotions then your conversion rate is more likely to be the interesting metric to monitor.
​

When running a variant test enough people must visit your website and take part in the test. This is because variant tests need to have collected enough data to ensure that the result of the experiment could not have occurred purely by chance. When a variant test has collected enough data that the results can be trusted it has reached ‘statistical significance’. 

Your statistical significance level reflects your risk tolerance and confidence level. For example, if you run a variant test experiment with a significance level of 95%, this means that if you determine a winner, you can be 95% confident that the observed results are real and not an error caused by randomness. It also means that there is a 5% chance that you could be wrong.

Statistical significance is important because it gives you confidence that the changes you make to your website or app do actually have a positive impact on your conversion rate and other metrics. Your metrics and numbers can fluctuate wildly from day to day, and statistical analysis provides a sound mathematical foundation for making business decisions and eliminating false positives.

We recommend using an online statistical significance calculator to determine the results of your variant tests: https://abtestguide.com/calc/