If you prefer, you can use the connector via Treasure Data CLI.
Open a terminal and run the following commands to install the newest TD Toolbelt.
The configuration file includes an in: section where you specify what comes into the connector from your integration and an out: section where you specify what the connector puts out to the database in Treasure Data.
Prepare configuration file (for example, config.yml) as shown in the following example. Provide your integration account access information.
in:
type: mail_publisher_smart
site_id: mysite
login_id: foo
password: bar
data_type: list_status
file_ids: [1,2,3]
filters:
- type: add_time
from_value:
mode: upload_time
to_column:
name: time
out: type: tdSpecify site_id, login_id and password. You can obtain the credentials from the MailPublisher's console.
Set list_status to data_type, then specify file_ids as list of file ID.
Set article_status or click_count_list to data_type, then specify start_date and duration. Set incremental if you want incremental loading.
in:
type: mail_publisher_smart
data_type: article_status
# data_type: click_count_list
start_date: 2018-10-01
duration: 1
incremental: trueRun td connector:preview command to validate your configuration file
$ td connector:preview config.ymlConnector sessions need at least one timestamp column in result data to be used as data partition key and the first timestamp column is chosen as the key by default. Because the MailPublisher connector doesn't have any timestamp column in the result schema, append the "time" column by adding the filter configuration as follows.
in:
type: mail_publisher_smart
...
filters:
- type: add_time
from_value:
mode: upload_time
to_column:
name: time
out:
type: tdRun td connector:create.
With the following command, a daily import session with MailPublisher connector is created.
$ td connector:create daily_mail_publisher_import \
"10 0 * * *" td_sample_db td_sample_table config.yml