Skip to content
Last updated

Dotdigital Export Integration

Dotdigital Engagement Cloud is an omnichannel marketing solution that sends data-triggered campaigns across email, SMS, social media, push notifications, and more. You can write job results from Treasure Data directly to Dotdigital.

Prerequisites

  • Basic knowledge of Treasure Data, including the TD Toolbelt.
  • An account in Dotdigital.
  • An API account in Dotdigital.

Limitations

Obtain Your API Account

  1. Log in to DotDigital. Navigate to https://login.dotdigital.com/login.aspx?ReturnUrl=/.

  2. Navigate to https://r1-app.dotdigital.com/access/api and select the API users tab**.**

  3. Select New user. DotDigital provides you with an email address. Complete the form by adding and confirming a password; a description is optional.

  4. Click Save.

Create a New Connection with TD Console

In Treasure Data, you must create and configure the data connection for export before running your query. As part of the data connection, you provide authentication to access the integration.

  1. Open the TD Console.

  2. Navigate to the Integrations Hub > Catalog.

  3. Click the search icon on the far right of the Catalog screen, and enter dotdigital.

  4. Hover over the dotdigital connector and select Create Authentication.

  5. The New Authentication dialog opens.

  6. Select your Region. The API User Name and API User Password fields auto-populate.

  7. Select Continue.

  8. Type a name for your connection.

  9. Select Done.

Configure Export Results in Your Data Connection

Next, you need to create or reuse a query. In the query, you configure the data connection. You might need to define the column mapping in the query.

Configure the Connection by Specifying the Parameters

  1. Open TD Console.

  2. Navigate to Data Workbench > Queries.

  3. Select the query that you plan to use to export data.

  4. Run the query.

  5. Select Export Results, located at the top of your query editor.

    The Choose Integration dialog opens.

  6. You have two options when selecting a connection to use to export the results:

  • Use an existing connection:

    • Type the connection name in the search box to filter.
    • Select your connection.
  • Create a new Dotdigital connection.

    • Complete the field values to create a new connection.

Type the required credentials for your new Dotdigital connection. Set the following parameters.

ParameterRequirementDescription
Export ModeRequiredThere are three different ways to move data: - Transactional Email (default). Trigger send emails for the triggered campaign in Dotdigital: send one email for each record. - Bulk Contacts: Upload records as contacts to Dotdigital. - Unified Contacts: Upload/Delete records as contacts to Dotdigital. An enhanced version of Bulk Contacts that enable contacts and their dependencies to be uploaded/deleted in one go.

Export Mode for Transactional Email

ParameterRequirementDescription
Campaign IDRequiredThe ID of the triggered campaign.
JSON Personalization ColumnsOptionalCommas separate the JSON column names.
Ignore Invalid RecordsOptionalIgnore invalid records and complete the job.
  • Sample Configuration

Export Mode for Bulk Contacts

ParameterRequirementDescription
Export OperationRequiredExport operation in bulk contacts mode - Upload (default). Upload records. - Delete. Delete contact from address book by id
Add to Address BookOptionalAdd to address book.
Address Book   Identifier ModeOptionalAddress book Identifier Mode to interact with: - Id (default). - Name. Resolve id from name
Address Book IDOptionalAddress book ID to interact with. This parameter is required in following situations: - Export Operation is "Upload" and "Add to Address Book" is enabled - Export Operation is "Delete"
Create A New Address Book When Not FoundOptionalCreate a new address book with the name when no address book with the provided name is found.
New Address Book VisibilityOptionalAddress book Visibility - Private (default). - Public.
Address Book NameOptionalAddress book Name to interact with. This parameter is required in the following situations: - Export Operation is "Upload",  and "Add to Address Book" is enabled - Export Operation is "Delete"
Merge OptionOptionalThe following merge options are available: - None (default). Do not merge. - Update All. Update all data fields - update_all - Ignore Blank. Ignores the blank values in the query result set - ignore_blank - Skip Existing Non-Empty Fields. Do not update data fields that contain values - skip_fields
Ignore Invalid RecordsOptionalIgnore invalid records and complete the job.
Enable Parallel UploadOptionalRun the job in parallel mode.
Thread CountOptionalNumber of threads run in parallel mode.
  • Sample Configuration

Export Mode for Unified Contacts

ParameterRequirementDescription
Export OperationRequiredExport operation in unified contacts mode - Upload (default). Upload records as unified contacts. - Delete. Delete contacts (hard delete)
Add to List (former Address Book)OptionalAdd contacts to a list.
List Identifier ModeOptionalList Identifier Mode to interact with: - Id (default). - Name. Resolve id from name
List IDOptionalList ID to interact with. This parameter is required in following situations: - Export Operation is "Upload" and "Add to List" is enabled - Export Operation is "Delete"
Create A New List When Not FoundOptionalCreate a new list with the name when no list with the provided name is found.
New List VisibilityOptionalList Visibility - Private (default). - Public.
List NameOptionalList Name to interact with. This parameter is required in the following situations: - Export Operation is "Upload",  and "Add to List" is enabled - Export Operation is "Delete"
Merge OptionOptionalThe following merge options are available: - overwrite (default). always overwrite all fields. - overwriteIfEmpty. overwrite empty fields only. - overwriteIfNotEmpty. overwrite if new values are not empty.
Ignore Invalid RecordsOptionalIgnore invalid records and complete the job.
  • Sample Configuration

Upload Operation

Delete Operation

Example of a Query to Populate Dotdigital

From Treasure Data, run the following query and export results into a connection for Dotdigital. Transactional emails use case-sensitive column names, while bulk contacts use case-insensitive column names.

  • Transactional Emails Query
SELECT an_email_column AS email,
another_first_name AS FIRSTNAME
another_first_name AS LASTNAME
FROM your_table;
  • Bulk Contacts Upload Query
SELECT an_email_column AS email,
another_first_name AS firstname
another_first_name AS lastname
FROM your_table;
  • Bulk Contacts Delete Query

SELECT contact_id AS id FROM your_table;

  • Unified Contacts Query

Upload Operation

SELECT match_identifier, contact_id, email, mobile_number, channel_properties, consent_records, firstname, lastname FROM your_table;

Delete Operation

SELECT match_identifier, contact_id, email, mobile_number FROM your_table;

Data Model

ColumnsRequireData TypeNotes
match_identifieryesstring- Values: contactId, email, mobileNumber - Case sensitive
contact_id- Require one of these identifier depends on value of match_identifier column - Leftover columns are optional.int64
emailstring
mobile_numberstring
channel_propertiesnostring in json format
preferencesnostring in json format
consent_recordsnostring in json format
any other columnsnostringmapped to datafields

(Optional) Schedule Query Export Jobs

You can use Scheduled Jobs with Result Export to periodically write the output result to a target destination that you specify.

Treasure Data's scheduler feature supports periodic query execution to achieve high availability.

When two specifications provide conflicting schedule specifications, the specification requesting to execute more often is followed while the other schedule specification is ignored.

For example, if the cron schedule is '0 0 1 * 1', then the 'day of month' specification and 'day of week' are discordant because the former specification requires it to run every first day of each month at midnight (00:00), while the latter specification requires it to run every Monday at midnight (00:00). The latter specification is followed.

Scheduling your Job Using TD Console

  1. Navigate to Data Workbench > Queries

  2. Create a new query or select an existing query.

  3. Next to Schedule, select None.

  4. In the drop-down, select one of the following schedule options:

    Drop-down ValueDescription
    Custom cron...Review Custom cron... details.
    @daily (midnight)Run once a day at midnight (00:00 am) in the specified time zone.
    @hourly (:00)Run every hour at 00 minutes.
    NoneNo schedule.

Custom cron... Details

Cron ValueDescription
0 * * * *Run once an hour.
0 0 * * *Run once a day at midnight.
0 0 1 * *Run once a month at midnight on the morning of the first day of the month.
""Create a job that has no scheduled run time.
 *    *    *    *    *
 -    -    -    -    -
 |    |    |    |    |
 |    |    |    |    +----- day of week (0 - 6) (Sunday=0)
 |    |    |    +---------- month (1 - 12)
 |    |    +--------------- day of month (1 - 31)
 |    +-------------------- hour (0 - 23)
 +------------------------- min (0 - 59)

The following named entries can be used:

  • Day of Week: sun, mon, tue, wed, thu, fri, sat.
  • Month: jan, feb, mar, apr, may, jun, jul, aug, sep, oct, nov, dec.

A single space is required between each field. The values for each field can be composed of:

Field ValueExampleExample Description
A single value, within the limits displayed above for each field.
A wildcard '*' to indicate no restriction based on the field.'0 0 1 * *'Configures the schedule to run at midnight (00:00) on the first day of each month.
A range '2-5', indicating the range of accepted values for the field.'0 0 1-10 * *'Configures the schedule to run at midnight (00:00) on the first 10 days of each month.
A list of comma-separated values '2,3,4,5', indicating the list of accepted values for the field.0 0 1,11,21 * *'Configures the schedule to run at midnight (00:00) every 1st, 11th, and 21st day of each month.
A periodicity indicator '*/5' to express how often based on the field's valid range of values a schedule is allowed to run.'30 */2 1 * *'Configures the schedule to run on the 1st of every month, every 2 hours starting at 00:30. '0 0 */5 * *' configures the schedule to run at midnight (00:00) every 5 days starting on the 5th of each month.
A comma-separated list of any of the above except the '*' wildcard is also supported '2,*/5,8-10'.'0 0 5,*/10,25 * *'Configures the schedule to run at midnight (00:00) every 5th, 10th, 20th, and 25th day of each month.
  1. (Optional) You can delay the start time of a query by enabling the Delay execution.

Execute the Query

Save the query with a name and run, or just run the query. Upon successful completion of the query, the query result is automatically exported to the specified destination.

Scheduled jobs that continuously fail due to configuration errors may be disabled on the system side after several notifications.

(Optional) You can delay the start time of a query by enabling the Delay execution.

Activate a Segment in Audience Studio

You can also send segment data to the target platform by creating an activation in the Audience Studio.

  1. Navigate to Audience Studio.
  2. Select a parent segment.
  3. Open the target segment, right-mouse click, and then select Create Activation.
  4. In the Details panel, enter an Activation name and configure the activation according to the previous section on Configuration Parameters.
  5. Customize the activation output in the Output Mapping panel.

  • Attribute Columns
    • Select Export All Columns to export all columns without making any changes.
    • Select + Add Columns to add specific columns for the export. The Output Column Name pre-populates with the same Source column name. You can update the Output Column Name. Continue to select + Add Columnsto add new columns for your activation output.
  • String Builder
    • + Add string to create strings for export. Select from the following values:
      • String: Choose any value; use text to create a custom value.
      • Timestamp: The date and time of the export.
      • Segment Id: The segment ID number.
      • Segment Name: The segment name.
      • Audience Id: The parent segment number.
  1. Set a Schedule.

  • Select the values to define your schedule and optionally include email notifications.
  1. Select Create.

If you need to create an activation for a batch journey, review Creating a Batch Journey Activation.

For Dotdigital, Audience Studio supports fetching IDs of Address Books, user can select a Address Book from the list. Only when the fetch is not success, manual typing is expected.

Configure Export Results in Workflow

Example Workflow Exporting Transactional Email

timezone: UTC

_export:
  td:
    database: sample_datasets

+export_to_dotdigital:
  td>: queries/sample.sql
  result_connection: my_dotdigital_output
  result_settings:
    campaign_id: 119178
    mode: transactional_email
    ignore_invalid_records: true
    json_columns: ORDER, ITEMS

Example Workflow Exporting Bulk Contacts Upload

timezone: UTC

_export:
  td:
    database: sample_datasets

+export_to_dotdigital:
  td>: queries/sample.sql
  result_connection: my_dotdigital_output
  result_settings:
    campaign_id: 119178
    mode: bulk_contacts
    ignore_invalid_records: true
    merge_option: update_all
    add_to_address_book: true
    address_book_id: 119178
    export_operation: upload
    address_book_name: "address_book_name"
    address_book_identifier_mode: name
    create_when_not_found: true
    new_address_book_visibility: private

Example Workflow Exporting Bulk Contacts Delete

timezone: UTC

_export:
  td:
    database: sample_datasets

+export_to_dotdigital:
  td>: queries/sample.sql
  result_connection: my_dotdigital_output
  result_settings:
    campaign_id: 119178
    mode: bulk_contacts
    ignore_invalid_records: true
    address_book_id_delete: 119178
    export_operation: delete
    address_book_name_delete: "address_book_name"
    address_book_identifier_mode_delete: name

Example Workflow Exporting Unified Contacts Upload

timezone: UTC

_export:
  td:
    database: sample_datasets

+export_to_dotdigital:
  td>: queries/sample.sql
  result_connection: my_dotdigital_output
  result_settings:
    mode: unified_contacts
    ignore_invalid_records: true
    unified_cts_export_operation: upload
    unified_cts_merge_option: overwrite
    unified_cts_add_to_list: true
    unified_cts_list_identifier_mode: id
    unified_cts_list_id: 119178

Example Workflow Exporting Unified Contacts Delete

timezone: UTC

_export:
  td:
    database: sample_datasets

+export_to_dotdigital:
  td>: queries/sample.sql
  result_connection: my_dotdigital_output
  result_settings:
    mode: unified_contacts
    ignore_invalid_records: true
    unified_cts_export_operation: delete

For more information on using integrations in a workflow to export data, see Using Workflows to Export Data with TD Toolbelt.