Marketing teams run A/B tests to test different types of campaigns for different groups, compare the performance of campaigns between groups, identify the best-performing marketing strategies and optimize future campaigns based on those insights. A/B Testing is now supported in Treasure Data Audience Studio, enabling you to:

  • Randomly split customers into groups of up to 5 to compare against a control group.
  • Run different types of campaigns for different groups by configuring an activation for each of them with different parameters
  • Compare the performance of campaigns between the groups using 3rd party integrations
  • Identify the best-performing marketing strategies for your customers
  • Optimize future campaigns based on insights using TD queries, workflows, and Treasure Insights to analyze the performance

This feature is only available to legacy customers. Contact your Customer Success Representative for more information.

Limitations

  • This feature supports group creation only.
  • You must contact Customer Success to have this feature enabled.
  • A/B test names must be unique. They cannot be the same as any existing or deleted A/B test.
  • When a profile in an A/B test segment is moved out of its Parent Segment, that profile drops out of the A/B test segment. This results in the A/B test segment profile numbers changing.
  • You can add a profile to an A/B test segment only while creating the segment.

Setting Up A/B Test Variants

1. Open TD Console.
2. Navigate to Audience Studio and select the parent segment and segment for which you want to set up the A/B test.
3. Right-click on the segment and select Create A/B test
4. Edit the details in the Create A/B Test modal. Select Create.
ParameterDescription
A/B test name

Name of the A/B test

The test name must only contain alphanumeric characters, spaces, and _.

After the test segments have been created, you can rename them using other characters (i.e. Japanese)

Number of variantsbetween 1-5
% of total to testPercentage of the segment you want to test 1-99% (evenly between the variants)
Stable ID field

The value of this attribute determines the assignment.

i.e. td_client_id

4.  A folder for the A/B test is created in the tree. The name is <original batch segment name>-<A/B test name>.
5.  Open the folder, and you can see the test variant segments with the names showing as <variant>-<A/B test name>.
6. Create rules for each variant using the segment editor to create segment rules.
7.  Activations can be created for each variant to test different campaigns.
  • No labels