| Table of Contents |
|---|
Install ‘td’ Command v0.11.9 or Later
You can install the newest Treasure Data Toolbelt.
| Code Block | ||
|---|---|---|
| ||
$ td --version 0.15.3 |
Create Configuration File
Prepare configuration file (for eg: load.yml) as shown in the following example, with your Salesforce Marketing Cloud account access information.
...
client_id: Salesforce Marketing Cloud client id.client_secret: Salesforce Marketing Cloud client secret.target: Salesforce Marketing Cloud entity object to be imported.
Optionally Preview Data to Import
You can preview data to be imported using the command td connector:preview.
| Code Block | ||
|---|---|---|
| ||
$ td connector:preview load.yml +-----------------+---------------------+--------------------+---- | id:long | name:string | description:string | ... +-----------------+---------------------+--------------------+---- | 42023 | "Hello" | apps | | 42045 | "World" | apps | +-----------------+---------------------+--------------------+---- |
Execute Load Job
Submit the load job. It may take a couple of hours depending on the data size. Users need to specify the database and table where their data are stored.
...
You can assign Time Format column to the "Partitioning Key" by "--time-column" option. |
Scheduled Execution
You can schedule periodic data connector execution for periodic Salesforce Marketing Cloud import. We configure our scheduler carefully to ensure high availability. By using this feature, you no longer need a cron daemon on your local data center.
Create the Schedule
A new schedule can be created using the td connector:create command. The name of the schedule, cron-style schedule, the database and table where their data will be stored, and the Data Connector configuration file are required.
...
By default, the schedule is setup in the UTC timezone. You can set the schedule in a timezone using -t or --timezone option. The `--timezone` option only supports extended timezone formats like 'Asia/Tokyo', 'America/Los_Angeles' etc. Timezone abbreviations like PST, CST are *not* supported and may lead to unexpected schedules. |
List the Schedules
You can see the list of currently scheduled entries by td connector:list.
| Code Block | ||
|---|---|---|
| ||
$ td connector:list
+-----------------------------------------+--------------+----------+-------+--------------+-----------------+----------------------------------------------+
| Name | Cron | Timezone | Delay | Database | Table | Config |
+-----------------------------------------+--------------+----------+-------+--------------+-----------------+----------------------------------------------+
| daily_salesforce_marketing_cloud_import | 10 0 * * * | UTC | 0 | td_sample_db | td_sample_table | {"type"=>"salesforce_marketing_cloud_v2", ... } |
+-----------------------------------------+--------------+----------+-------+--------------+-----------------+----------------------------------------------+
|
Show the Setting and History of Schedules
td connector:show shows the execution setting of a schedule entry.
...
| Code Block | ||
|---|---|---|
| ||
% td connector:history daily_salesforce_marketing_cloud_import +--------+---------+---------+--------------+-----------------+----------+---------------------------+----------+ | JobID | Status | Records | Database | Table | Priority | Started | Duration | +--------+---------+---------+--------------+-----------------+----------+---------------------------+----------+ | 578066 | success | 10000 | td_sample_db | td_sample_table | 0 | 2015-04-18 00:10:05 +0000 | 160 | | 577968 | success | 10000 | td_sample_db | td_sample_table | 0 | 2015-04-17 00:10:07 +0000 | 161 | | 577914 | success | 10000 | td_sample_db | td_sample_table | 0 | 2015-04-16 00:10:03 +0000 | 152 | | 577872 | success | 10000 | td_sample_db | td_sample_table | 0 | 2015-04-15 00:10:04 +0000 | 163 | | 577810 | success | 10000 | td_sample_db | td_sample_table | 0 | 2015-04-14 00:10:04 +0000 | 164 | | 577766 | success | 10000 | td_sample_db | td_sample_table | 0 | 2015-04-13 00:10:04 +0000 | 155 | | 577710 | success | 10000 | td_sample_db | td_sample_table | 0 | 2015-04-12 00:10:05 +0000 | 156 | | 577610 | success | 10000 | td_sample_db | td_sample_table | 0 | 2015-04-11 00:10:04 +0000 | 157 | +--------+---------+---------+--------------+-----------------+----------+---------------------------+----------+ 8 rows in set |
Delete the Schedule
td connector:delete removes the schedule.
| Code Block | ||
|---|---|---|
| ||
$ td connector:delete daily_salesforce_marketing_cloud_import |
Incremental Loading
For Data Extensions
Treasure Data supports incremental loading for Data Extensions that have a date field.
...
If incremental: false is set, The data connector loads all records for the target specified. This is one-time activity.
For Email Events
Treasure Data supports incremental loading for Email Events based on their event date.
...
If incremental: false is set, The data connector loads all records for the target specified. This is a one-time activity.
Modes for the Out Plugin
| Excerpt Include | ||||||
|---|---|---|---|---|---|---|
|
Sample Configuration for Each Target
For Campaign target using Legacy Package
...
Parameters | Description | Default value |
|---|---|---|
type | must be salesforce_marketing_cloud_v2 | |
client_id | Salesforce Marketing Cloud | |
client_secret | Salesforce Marketing Cloud | |
auth_type | Package Type This is a enum (v1, v2) where v1 is Legacy package and v2 is Enhanced Package | v1 |
tenant_auth_uri | Tenant Authentication Base URI (only enter this option if auth_type = v1) This will require since is required from September 2022 as Marketing Cloud is going to deprecate the legacy endponts [https://*.exacttargetapis.com]. For more information please check : https://help.salesforce.com/s/articleView?id=000356497, https://help.salesforce.com/s/articleView?id=000356498 | |
| auth_uri | Authentication Base URI (only enter this option if auth_type = v2) |
...
Parameters | Description | Default value |
|---|---|---|
type | must be salesforce_marketing_cloud_v2 | |
client_id | Salesforce Marketing Cloud | |
client_secret | Salesforce Marketing Cloud | |
auth_type | Package Type This is a enum (v1, v2) where v1 is Legacy package and v2 is Enhanced Package | v1 |
tenant_auth_uri | Tenant Authentication Base URI (only enter this option if auth_type = v1) This will require since is required as of September 2022 as Marketing Cloud is going to deprecate the legacy endponts [https://*.exacttargetapis.com]. For more information please check : https://help.salesforce.com/s/articleView?id=000356497, https://help.salesforce.com/s/articleView?id=000356498 | |
| auth_uri | Authentication Base URI (only enter this option if auth_type = v2) | |
contact_attributes_request_limit | Number of attributes to ingest for each contact in a single request. When number of attributes greater than this value, it is split into multiple requests | 100 |
contact_multiple_requests | Enable plugin to ingest contact detail with so many attributes in multiple requests (This only effects when number of attributes greater than 100 ) | false |
ignore_attribute_set_names | List of attributes are ignored. Useful when you want to eliminate some unnecessary attributes | null |
...
Parameters | Description | Default value |
|---|---|---|
type | must be salesforce_marketing_cloud_v2 | |
client_id | Salesforce Marketing Cloud | |
client_secret | Salesforce Marketing Cloud | |
auth_type | Package Type This is a enum (v1, v2) where v1 is Legacy package and v2 is Enhanced Package | v1 |
tenant_auth_uri | Tenant Authentication Base URI (only enter this option if auth_type = v1) This will require since is required as of September 2022 as Marketing Cloud is going to deprecate the legacy endponts [https://*.exacttargetapis.com]. For more information please check : https://help.salesforce.com/s/articleView?id=000356497, https://help.salesforce.com/s/articleView?id=000356498 | |
| auth_uri | Authentication Base URI (only enter this option if auth_type = v2) | |
shared_data_extension | Set this flag to true when you want to ingest the data from shared data extensions | false |
incremental | Set this flag to true if you want to get newer data between each schedule only | false |
incremental_column_name | Column used for incremental loading. Must be set if you set incremental = true | null |
from_date | Set this to ingest data from this date | null |
fetch_days | Duration to ingest is from_date to from_date + fetch_days | 1 |
...
Parameters | Description | Default value |
|---|---|---|
type | must be salesforce_marketing_cloud_v2 | |
client_id | Salesforce Marketing Cloud | |
client_secret | Salesforce Marketing Cloud | |
auth_type | Package Type This is a enum (v1, v2) where v1 is Legacy package and v2 is Enhanced Package | v1 |
tenant_auth_uri | Tenant Authentication Base URI (only enter this option if auth_type = v1) This will require since is required as of September 2022 as Marketing Cloud is going to deprecate the legacy endponts [https://*.exacttargetapis.com]. For more information please check : https://help.salesforce.com/s/articleView?id=000356497, https://help.salesforce.com/s/articleView?id=000356498 | |
| auth_uri | Authentication Base URI (only enter this option if auth_type = v2) | |
target | must be email_event (in this case) | |
search_term | Name of email you want to ingest | null |
search_mode | Mode for name matching pattern. Must be exact or partial only | exact |
email_event_types | List of event types you want to import. Valid options are as follows:
| null |
start_time | Set this to ingest data from this time | null |
end_time | Set this to ingest data to this time | null |
incremental | Set this flag to true if you want to get newer data between each schedule only | false |
...