> For the complete documentation index, see [llms.txt](https://help.smartico.ai/welcome/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://help.smartico.ai/welcome/products/crm-automation/a-b-testing.md).
# A/B Testing
This document describes the specifics of A/B Testing and how to use it with Smartico.
A/B testing is a method used to compare two versions of a marketing asset to determine which one performs better. In this process, the audience is divided into two groups, and each group is shown a different version (Version A and Version B) of the marketing asset.
With Smartico, you can conduct A/B testing across various communication channels including email, pop-up, inbox, SMS, and push notifications. The purpose is to determine which version generates a better response in terms of the desired outcome, such as positive interactions (e.g. deposits) and conversion rate for example.
To do an A/B testing for a specific marketing asset you need to start by preparing the desired resources and the variations for it. For example, if you want to compare 2 versions of pop-ups that promote a 10% cashback bonus on deposit you should first create 2 different pop-ups and then you can easily attach them to the campaign.
When creating the campaign flow, you can simply add the different versions of the asset, in our case the two versions of the pop-ups that you want to compare. You can choose the percentage of the audience who will receive Variant A and who will receive Variant B.
You can test multiple variations (two or more) within the same communication node, provided the total percentage across all variations equals 100%.
### In-Asset Testing vs. Split Traffic Node
Smartico provides two distinct splitting mechanisms depending on your testing goal:
* **In-Asset Communication A/B Testing (Single Node):** Used to compare multiple template variations within the same communication channel at a single step in the flow. It supports automated conversion tracking against an A/B target and automated winner selection after reaching a message threshold.
* **Split Traffic Flow Node:** A dedicated flow-control activity that splits incoming users across two or more separate journey branches according to defined percentage splits. Unlike in-asset testing, the Split Traffic node does not automatically select a winning branch; it permanently routes traffic between different journey paths, allowing you to test completely different communication sequences, delays, or reward strategies.
The **‘Number of message to send before selecting winner of A/B test’** option (default: 1,000) allows you to define how many messages (emails, SMS, pop-ups, push notifications, etc.) must be sent across all variations before the system evaluates and selects the winning variation. For example, setting this threshold to 5,000 means the system will distribute variations by the configured percentages until 5,000 messages have been sent, after which it evaluates conversion performance and routes all subsequent users to the winning asset.
{% hint style="info" %}
Keep in mind that if you are using the A/B testing in a Scheduled Campaign, the campaign should be recurring to determine the exact winner and to start sending only the winner. If the campaign is scheduled for one time the winner cannot be defined and the variations will be split the way they are defined in the communication asset.
{% endhint %}
The winner of the A/B Testing is defined automatically based on how many users reached the A/B target. You can mark any part of the flow as an A/B target, when it is set you can do an A/B analysis towards the targeted step.
An automated background evaluation process checks campaign performance on an hourly basis. Once total sends exceed the configured message threshold, the variation with the highest conversion rate (conversions to the A/B target divided by total entries) is selected as the winner. In the flow editor, the winning variation is badged with a checkmark showing its conversion statistics, and all subsequent users entering that activity will receive only the winning variation. If the message threshold is later increased, the winner lock is cleared and traffic splitting resumes until the new limit is met.
The results of the A/B testing can be found in the **A/B Tests Analysis** report in the Back-Office. In the report, you can view the total audience entries, the number of converted users who reached the A/B target, and the resulting conversion ratio for each variation.
### Frequently Asked Questions
**How is the winner of an A/B test calculated?**
The winner is determined strictly by the conversion rate to the downstream step marked with the **A/B target** switch (calculated as users reaching the target divided by total entries for that variation). It is not determined by open rate or click-through rate unless a click/interaction step is specifically configured as the A/B target.
**What happens if a campaign does not reach the message threshold?**
If the total sent messages do not reach the threshold (such as in low-volume campaigns or one-time scheduled blasts), the system continues to split traffic according to the configured percentages for every user. No automated winner is selected, but you can analyze the comparative conversion numbers manually in the A/B Tests Analysis report.
**Can I run permanent random message rotation without selecting a winner?**
Yes. If you want ongoing random distribution across variations without locking into a winner, either set the message threshold to an exceptionally high number (up to 100,000,000) so it is never triggered, or use the **Split Traffic** activity node to distribute traffic between different message branches continuously.
**Can I test more than two variations in a single message activity?**
Yes. You can add three or more variations (A, B, C, etc.) to any supported communication activity. The only requirement is that the sum of the percentage distribution across all variations must equal exactly 100%.
**Which communication channels support in-asset A/B testing?**
In-asset A/B testing is supported across Email, Pop-up, Push notifications, Inbox messages, SMS, IVR, and instant messaging channels including Telegram, WhatsApp, Viber, and RCS.
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://help.smartico.ai/welcome/products/crm-automation/a-b-testing.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
