Marketing teams run A/B tests to test different types of campaigns for different groups, compare the performance of campaigns between groups, identify the best-performing marketing strategies and optimize future campaigns based on those insights. A/B Testing is now supported in Treasure Data Audience Studio, enabling you to:

Randomly split customers into groups of up to 5 to compare against a control group.
Run different types of campaigns for different groups by configuring an activation for each of them with different parameters
Compare the performance of campaigns between the groups using 3rd party integrations
Identify the best-performing marketing strategies for your customers
Optimize future campaigns based on insights using TD queries, workflows, and Treasure Insights to analyze the performance

This feature is only available to legacy customers. Contact your Customer Success Representative for more information.

Limitations

This feature supports group creation only.
You must contact Customer Success to have this feature enabled.
A/B test names must be unique. They cannot be the same as any existing or deleted A/B test.
When a profile in an A/B test segment is moved out of its Parent Segment, that profile drops out of the A/B test segment. This results in the A/B test segment profile numbers changing.
You can add a profile to an A/B test segment only while creating the segment.

Setting Up A/B Test Variants

1. Open TD Console.

2. Navigate to Audience Studio and select the parent segment and segment for which you want to set up the A/B test.

3. Right-click on the segment and select Create A/B test.
4. Edit the details in the Create A/B Test modal. Select Create.

Parameter	Description
A/B test name	Name of the A/B test The test name must only contain alphanumeric characters, spaces, and _. After the test segments have been created, you can rename them using other characters (i.e. Japanese)
Number of variants	between 1-5
% of total to test	Percentage of the segment you want to test 1-99% (evenly between the variants)
Stable ID field	The value of this attribute determines the assignment. i.e. td_client_id

Page tree

Limitations

Setting Up A/B Test Variants

1. Open TD Console.

2. Navigate to Audience Studio and select the parent segment and segment for which you want to set up the A/B test.

3. Right-click on the segment and select Create A/B test.
4. Edit the details in the Create A/B Test modal. Select Create.

4. A folder for the A/B test is created in the tree. The name is <original batch segment name>-<A/B test name>.

5. Open the folder, and you can see the test variant segments with the names showing as <variant>-<A/B test name>.

6. Create rules for each variant using the segment editor to create segment rules.

7. Activations can be created for each variant to test different campaigns.

Page tree

Setting Up A B Testing

Limitations

Setting Up A/B Test Variants

1. Open TD Console.

2. Navigate to Audience Studio and select the parent segment and segment for which you want to set up the A/B test.

3. Right-click on the segment and select Create A/B test. 4. Edit the details in the Create A/B Test modal. Select Create.

4. A folder for the A/B test is created in the tree. The name is <original batch segment name>-<A/B test name>.

5. Open the folder, and you can see the test variant segments with the names showing as <variant>-<A/B test name>.

6. Create rules for each variant using the segment editor to create segment rules.

7. Activations can be created for each variant to test different campaigns.

3. Right-click on the segment and select Create A/B test.
4. Edit the details in the Create A/B Test modal. Select Create.