◾A/B Testing

This document describes the specifics of A/B Testing and how to use it with Smaritco.

A/B testing is a method used to compare two versions of a marketing asset to determine which one performs better. In this process, the audience is divided into two groups, and each group is shown a different version (Version A and Version B) of the marketing asset.

With Smartico, you can conduct A/B testing across various communication channels including email, pop-up, inbox, SMS, and push notifications. The purpose is to determine which version generates a better response in terms of the desired outcome, such as positive interactions (e.g. deposits) and conversion rate for example.

To do an A/B testing for a specific marketing asset you need to start by preparing the desired resources and the variations for it. For example, if you want to compare 2 versions of pop-ups that promote a 10% cashback bonus on deposit you should first create 2 different pop-ups and then you can easily attach them to the campaign.

When creating the campaign flow, you can simply add the different versions of the asset, in our case the two versions of the pop-ups that you want to compare. You can choose the percentage of the audience who will receive Variant A and who will receive Variant B.

The ‘Number of the messages to send before selecting the winner’ option gives you the possibility to define how many pop-ups (emails, SMSs, inbox messages, etc.) will be sent before selecting the winner of the A/B test. For example, the winner from the comparison between the 2 variants of the pop-ups can be defined after 5000 pop-ups are sent.

Keep in mind that if you are using the A/B testing in a Scheduled Campaign, the campaign should be recurring to determine the exact winner and to start sending only the winner. If the campaign is scheduled for one time the winner cannot be defined and the variations will be split the way they are defined in the communication asset.

The winner of the A/B Testing is defined automatically based on how many users reached the A/B target. You can mark any part of the flow as an A/B target, when it is set you can do an A/B analysis towards the targeted step.

The results of the A/B testing can be found in the A/B Tests Analisis in Smartico BO. In the report, you can see all the users who participated in the campaign, the converted users, and the conversion ratio.