# Configuring A/B Testing A/B testing helps marketers test treatments (for example, different marketing Calls to Action in an email) to determine which treatments are performing better. With Treasure Data's A/B Testing, you can compare one to eight versions of something to determine which performs better. To accomplish this test, at a minimum, you need one test case and a control group to validate the results. A control group is important because it provides a benchmark to measure performance against. A control group highlights what works as much as what doesn't. INFO The steps to configure A/B Testing apply only to draft journeys. ## Prerequisites - A parent segment. - A new draft journey or an existing draft journey. - Basic understanding of Treasure Data. ## Limitations - Each stage of a journey can only have two layers of an A/B test added. ## Verify the Status of a Journey You can verify the current status of a draft journey to which you want to add or update A/B tests. The following example does not include an A/B test because we are going to add one in this tutorial. In the case the journey included an existing A/B test, the request retrieves the name, variant, and allocation information for the specified journey. This example uses the following values | | | | --- | --- | | Method | GET | | Endpoint | https://api-cdp.treasuredata.com/entities/journeys/{journeyId} | | Header | `Authorization: TD1 ` | | Header | `Content-Type: application/json` | You can use a base URL that is optimal for your geographic location. For details, see [Treasure Data Sites and baseURLs](/apis/endpoints/endpoints). ```bash Sample Request export journeyId="117385" export API_KEY="" curl --location --request GET "https://api-cdp.treasuredata.com/entities/journeys/${journeyId}" \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --header "Authorization: TD1 ${API_KEY}" ``` ```json Sample Response { "data": { "id": "117385", "type": "journey", "attributes": { "audienceId": "57350", "name": "Purchase journey", "description": "", "state": "draft", "goal": null, "createdAt": "2023-09-11T20:47:29.086Z", "updatedAt": "2023-09-11T20:47:29.086Z", "launchedAt": null, "allowReentry": false, "paused": false, "pausedAt": null, "journeyStages": [ { "id": "149348", "name": "Awareness", "description": null, "steps": { "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7": { "type": "Activation", "next": "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03", "description": null, "name": "act_1", "journeyActivationStepId": "153668" }, "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03": { "type": "WaitStep", "next": null, "waitStep": 1, "waitStepUnit": "day" } }, "rootStep": "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7", "entryCriteria": { "name": "browse", "segmentId": "912056", "description": "" }, "milestone": null, "exitCriterias": null } ] }, "relationships": { "parentFolder": { "data": { "id": "183985", "type": "folder-segment" } }, "jumpFrom": { "data": [] }, "jumpTo": { "data": [] } } } } ``` About this request: - Replace `journeyId` with the draft journey to which you want to add or update A/B tests. - Replace `API_KEY` with your personal API key. About this response object: - The name of the A/B test and its variants are contained within the journey stage. ## Create A/B Tests in a Journey You add an A/B test to any decision point; there is no limit to the number of A/B tests within a journey. The following example sets up a test name, number of variants, percentage allocation to each variable, and a stable ID key. ```bash Sample Request export journeyId="117385" export API_KEY="" curl --location --request POST "https://api-cdp.treasuredata.com/entities/journeys/${journeyId}" \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --header "Authorization: TD1 ${API_KEY}" \ --data-raw '{ "type": "ABTest", "name": "A/B Test Name", "customizedSplit": true, "uniqueId": "cdp_customer_id", "variants": [ { "id": "ee9aa667-60c7-42e0-b8b1-5cee9a2edfb3", "name": "Variant A", "percentage": 45, "next": null }, { "id": "d4179b47-8d9a-495c-bddd-5dcf25c295c6", "name": "Variant B", "percentage": 5, "next": null }, { "id": "bbf9c797-efb0-41ae-83b0-53082d8337a8", "name": "Control", "percentage": 50, "next": null } ] }' ``` ```json Sample Response { "data": { "id": "117385", "type": "journey", "attributes": { "audienceId": "57350", "name": "Purchase journey", "description": "", "state": "draft", "goal": null, "createdAt": "2023-09-11T20:47:29.086Z", "updatedAt": "2023-09-11T20:47:29.086Z", "launchedAt": null, "allowReentry": false, "paused": false, "pausedAt": null, "journeyStages": [ { "id": "149348", "name": "Awareness", "description": null, "steps": { "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7": { "type": "Activation", "next": "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03", "description": null, "name": "act_1", "journeyActivationStepId": "153668" }, "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03": { "type": "WaitStep", "next": "e48c0c6f-7de2-4d94-9b22-c6c3eb51105e", "waitStep": 1, "waitStepUnit": "day" }, "e48c0c6f-7de2-4d94-9b22-c6c3eb51105e": { "type": "ABTest", "variants": [ { "id": "ee9aa667-60c7-42e0-b8b1-5cee9a2edfb3", "name": "Variant A", "percentage": 45, "next": null }, { "id": "d4179b47-8d9a-495c-bddd-5dcf25c295c6", "name": "Variant B", "percentage": 5, "next": null }, { "id": "bbf9c797-efb0-41ae-83b0-53082d8337a8", "name": "Control", "percentage": 50, "next": null } ], "name": "test", "customizedSplit": true, "uniqueId": "cdp_customer_id" } }, "rootStep": "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7", "entryCriteria": { "name": "browse", "segmentId": "912056", "description": "" }, "milestone": null, "exitCriterias": null } ] }, "relationships": { "parentFolder": { "data": { "id": "183985", "type": "folder-segment" } }, "createdBy": { "data": { "id": "1020", "type": "user" } }, "updatedBy": { "data": { "id": "1020", "type": "user" } }, "activations": { "data": [] }, "journeyActivationSteps": { "data": [ { "id": "153668", "type": "journeyActivationStep" } ] }, "jumpFrom": { "data": [] }, "jumpTo": { "data": [] } } } } ``` About this request: - Replace `journeyId` with the specified journey ID. - Replace `API_KEY` with your personal API key. About this response object: - The name of the A/B test and its variants are added to the journey stage. ## Modify an A/B Test You can make changes to an A/B test while it is in draft mode. For instance, you can rename the test or its variants, delete variants or the A/B test, change the Unique ID field, or the allocation percentage. ```bash Sample Request export journeyId="117385" export API_KEY="" curl --location --request PATCH "https://api-cdp.treasuredata.com/entities/journeys/${journeyId}" \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --header "Authorization: TD1 ${API_KEY}" \ --data-raw '{ "type": "ABTest", "name": "Sending promotion emails", "customizedSplit": false, "uniqueId": "cdp_customer_id", "variants": [ { "id": "51527fb6-72c4-4e85-a4bc-f3b7094100a2", "name": "Semi monthly", "percentage": 17, "next": null }, { "id": "d4179b47-8d9a-495c-bddd-5dcf25c295c6", "name": "Semi weekly", "percentage": 17, "next": null }, { "id": "ee9aa667-60c7-42e0-b8b1-5cee9a2edfb3", "name": "Weekly Once", "percentage": 17, "next": null }, { "id": "bbf9c797-efb0-41ae-83b0-53082d8337a8", "name": "Control", "percentage": 49, "next": null } ] }' ``` ```json Sample Response "data": { "id": "117385", "type": "journey", "attributes": { "audienceId": "57350", "name": "Purchase journey", "description": "", "state": "draft", "goal": null, "createdAt": "2023-09-11T20:47:29.086Z", "updatedAt": "2023-09-11T20:47:29.086Z", "launchedAt": null, "allowReentry": false, "paused": false, "pausedAt": null, "journeyStages": [ { "id": "149348", "name": "Awareness", "description": null, "steps": { "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7": { "type": "Activation", "next": "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03", "description": null, "name": "act_1", "journeyActivationStepId": "153668" }, "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03": { "type": "WaitStep", "next": "e48c0c6f-7de2-4d94-9b22-c6c3eb51105e", "waitStep": 1, "waitStepUnit": "day" }, "e48c0c6f-7de2-4d94-9b22-c6c3eb51105e": { "type": "ABTest", "variants": [ { "id": "51527fb6-72c4-4e85-a4bc-f3b7094100a2", "name": "Semi monthly", "percentage": 17, "next": null }, { "id": "d4179b47-8d9a-495c-bddd-5dcf25c295c6", "name": "Semi weekly", "percentage": 17, "next": null }, { "id": "ee9aa667-60c7-42e0-b8b1-5cee9a2edfb3", "name": "Weekly Once", "percentage": 17, "next": null }, { "id": "bbf9c797-efb0-41ae-83b0-53082d8337a8", "name": "Control", "percentage": 49, "next": null } ], "name": "Sending promotion emails", "customizedSplit": false, "uniqueId": "cdp_customer_id" } }, "rootStep": "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7", "entryCriteria": { "name": "browse", "segmentId": "912056", "description": "" }, "milestone": null, "exitCriterias": null } ] }, "relationships": { "parentFolder": { "data": { "id": "183985", "type": "folder-segment" } }, "createdBy": { "data": { "id": "1020", "type": "user" } }, "updatedBy": { "data": { "id": "1020", "type": "user" } }, "activations": { "data": [] }, "journeyActivationSteps": { "data": [ { "id": "153668", "type": "journeyActivationStep" } ] }, "jumpFrom": { "data": [] }, "jumpTo": { "data": [] } } } } ``` About this request: - Replace `journeyId` with journey you want to add or update A/B tests. - Replace `API_KEY` with your personal API key. About this response object: - The name of the A/B test and its variants have been modified. - The number of variants is increased in the response from 2 to 3. - The customized splits is changed to `false`. - The percentage allocated to each variant has also been modified. ## Delete an A/B Test You can delete an A/B test while it is in draft mode. The section contains: [Verify the A/B Test in the Journey Stage](#verify-the-ab-test-in-the-journey-stage) [Delete an A/B test Within a Journey Stage](#delete-an-ab-test-within-a-journey-stage) [Confirming Journey Status](#confirming-journey-status) ### Verify the A/B Test in the Journey Stage To confirm which A/B test to delete, you first need to confirm an A/B Test applied to a stage. In the example that follows, you can view the journey stage and A/B tests for journey `117385`. **Sample Request** ```bash Sample Request Verify A/B Test export journeyId="117385" export API_KEY="" curl --location --request GET "https://api-cdp.treasuredata.com/entities/journeys/${journeyId}" \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --header "Authorization: TD1 ${API_KEY}" ``` ```json Sample Response { "data": { "id": "117385", "type": "journey", "attributes": { "audienceId": "57350", "name": "Purchase journey", "description": "", "state": "draft", "goal": null, "createdAt": "2023-09-11T20:47:29.086Z", "updatedAt": "2023-09-11T20:47:29.086Z", "launchedAt": null, "allowReentry": false, "paused": false, "pausedAt": null, "journeyStages": [ { "id": "149348", "name": "Awareness", "description": null, "steps": { "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7": { "type": "Activation", "next": "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03", "description": null, "name": "act_1", "journeyActivationStepId": "153668" }, "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03": { "type": "WaitStep", "next": "e48c0c6f-7de2-4d94-9b22-c6c3eb51105e", "waitStep": 1, "waitStepUnit": "day" }, "e48c0c6f-7de2-4d94-9b22-c6c3eb51105e": { "type": "ABTest", "variants": [ { "id": "51527fb6-72c4-4e85-a4bc-f3b7094100a2", "name": "Semi monthly", "percentage": 17, "next": null }, { "id": "d4179b47-8d9a-495c-bddd-5dcf25c295c6", "name": "Semi weekly", "percentage": 17, "next": null }, { "id": "ee9aa667-60c7-42e0-b8b1-5cee9a2edfb3", "name": "Weekly Once", "percentage": 17, "next": null }, { "id": "bbf9c797-efb0-41ae-83b0-53082d8337a8", "name": "Control", "percentage": 49, "next": null } ], "name": "Sending promotion emails", "customizedSplit": false, "uniqueId": "cdp_customer_id" } }, "rootStep": "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7", "entryCriteria": { "name": "browse", "segmentId": "912056", "description": "" }, "milestone": null, "exitCriterias": null } ] }, "relationships": { "parentFolder": { "data": { "id": "183985", "type": "folder-segment" } }, "createdBy": { "data": { "id": "1020", "type": "user" } }, "updatedBy": { "data": { "id": "1020", "type": "user" } }, "activations": { "data": [] }, "journeyActivationSteps": { "data": [ { "id": "153668", "type": "journeyActivationStep" } ] }, "jumpFrom": { "data": [] }, "jumpTo": { "data": [] } } } } ``` About this request: An A/B test can be deleted from a journey stage by omitting the test in the same PATCH request. This request confirms the specified journey has an A/B Test in Stage 1. - Replace `journeyId` with journey you want to add or update A/B tests. - Replace `API_KEY` with your personal API key. ### Delete an A/B Test Within a Journey Stage To delete the A/B test, the information about the A/B test is omitted from the PATCH. In the example that follows, the A/B test and its associated variants are absent from the payload for journey `117385`. ```bash Sample Request - Omit the A/B Test export journeyId="117385" export API_KEY="" curl --location --request PATCH "https://api-cdp.treasuredata.com/entities/journeys/${journeyId}" \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --header "Authorization: TD1 ${API_KEY}" \ --data-raw '{ "id": "117385", "type": "journey", "attributes": { "audienceId": "57350", "name": "Purchase journey", "description": "", "state": "draft", "goal": null, "createdAt": "2023-09-11T20:47:29.086Z", "updatedAt": "2023-09-11T20:47:29.086Z", "launchedAt": null, "allowReentry": false, "paused": false, "pausedAt": null, "journeyStages": [ { "id": "149348", "name": "Awareness", "description": null, "steps": { "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7": { "type": "Activation", "next": "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03", "description": null, "name": "act_1", "journeyActivationStepId": "153668" }, "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03": { "type": "WaitStep", "next": null, "waitStep": 1, "waitStepUnit": "day" } }, "rootStep": "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7", "entryCriteria": { "name": "browse", "segmentId": "912056", "description": "" }, "milestone": null, "exitCriterias": null } ] }, "relationships": { "parentFolder": { "data": { "id": "183985", "type": "folder-segment" } }, "createdBy": { "data": { "id": "1020", "type": "user" } }, "updatedBy": { "data": { "id": "1020", "type": "user" } }, "activations": { "data": [] }, "journeyActivationSteps": { "data": [ { "id": "153668", "type": "journeyActivationStep" } ] }, "jumpFrom": { "data": [] }, "jumpTo": { "data": [] } } } ``` About this request: - The name of the A/B test and its variants have been removed from the PATCH request. - The "next" step is marked as "null" in the object preceding the A/B test. - Replace `journeyId` with journey from which you want to remove an A/B test. - Replace `API_KEY` with your personal API key. ```json Sample Response - Confirms absence of A/B Test { "data": { "id": "117385", "type": "journey", "attributes": { "audienceId": "57350", "name": "Purchase journey", "description": "", "state": "draft", "goal": null, "createdAt": "2023-09-11T20:47:29.086Z", "updatedAt": "2023-09-11T20:47:29.086Z", "launchedAt": null, "allowReentry": false, "paused": false, "pausedAt": null, "journeyStages": [ { "id": "149348", "name": "Awareness", "description": null, "steps": { "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7": { "type": "Activation", "next": "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03", "description": null, "name": "act_1", "journeyActivationStepId": "153668" }, "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03": { "type": "WaitStep", "next": null, "waitStep": 1, "waitStepUnit": "day" } }, "rootStep": "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7", "entryCriteria": { "name": "browse", "segmentId": "912056", "description": "" }, "milestone": null, "exitCriterias": null } ] }, "relationships": { "parentFolder": { "data": { "id": "183985", "type": "folder-segment" } }, "createdBy": { "data": { "id": "1020", "type": "user" } }, "updatedBy": { "data": { "id": "1020", "type": "user" } }, "activations": { "data": [] }, "journeyActivationSteps": { "data": [ { "id": "153668", "type": "journeyActivationStep" } ] }, "jumpFrom": { "data": [] }, "jumpTo": { "data": [] } } } } ``` About this response: - The response confirms the A/B test and its variants have been removed from the journey stage. ### Confirming Journey Status If additional confirmation apart from the response is required, you can do a GET to the endpoint `entities/journeys/{journey_id}`. A similar response as the PATCH request's response is displayed when a GET to the same journey in performed. The journey attributes do not display any A/B test information, as it was successfully deleted during the PATCH request. ```json Sample Response - Confirms Journey Status { "data": { "id": "117385", "type": "journey", "attributes": { "audienceId": "57350", "name": "Purchase journey", "description": "", "state": "draft", "goal": null, "createdAt": "2023-09-11T20:47:29.086Z", "updatedAt": "2023-09-11T20:47:29.086Z", "launchedAt": null, "allowReentry": false, "paused": false, "pausedAt": null, "journeyStages": [ { "id": "149348", "name": "Awareness", "description": null, "steps": { "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7": { "type": "Activation", "next": "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03", "description": null, "name": "act_1", "journeyActivationStepId": "153668" }, "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03": { "type": "WaitStep", "next": null, "waitStep": 1, "waitStepUnit": "day" } }, "rootStep": "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7", "entryCriteria": { "name": "browse", "segmentId": "912056", "description": "" }, "milestone": null, "exitCriterias": null } ] }, "relationships": { "parentFolder": { "data": { "id": "183985", "type": "folder-segment" } }, "createdBy": { "data": { "id": "1020", "type": "user" } }, "updatedBy": { "data": { "id": "1020", "type": "user" } }, "activations": { "data": [] }, "journeyActivationSteps": { "data": [ { "id": "153668", "type": "journeyActivationStep" } ] }, "jumpFrom": { "data": [] }, "jumpTo": { "data": [] } } } } ``` ## A/B Testing Statistics You can choose to show the number of profiles in a particular A/B test variant within the same parent segment using the following rule. ```json { "type": "And", "conditions": [ { "type": "Value", "exclude": true, "leftValue": { "journeyId": 1, "journeyStageId": 2, "location": "ab_test__variant_", "tense": "intime" }, "operator": { "type": "IsNull", "not": true } } ] } ```