Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

This feature is in Beta. For more information, contact support@treasuredata.com.

...

If the system detects your column name or type unexpectedly, modify load.yml directly and preview again.

Currently, the Data Connector supports parsing of "boolean", "long", "double", "string", "timestamp", and "json" types.

Execute Load Job

Finally, submit the load job. It might take a couple of hours depending on the data size. Users need to specify the database and table where their data is stored.

...

Code Block
linenumberstrue
$ td connector:issue load.yml --database td_sample_db --table td_sample_table --time-column created_at --auto-create-table


You can assign Time Format column to the "Partitioning Key" by "--time-column" option.

Scheduled Execution

You can schedule periodic Data Connector execution for periodic NetSuite import. We take great care in distributing and operating our scheduler in order to achieve high availability. By using this feature, you no longer need a cron daemon on your local data center.

...

Code Block
linenumberstrue
$ td connector:create \
    daily_netsuite_import \
    "10 0 * * *" \
    td_sample_db \
    td_sample_table \
    load.yml


The `cron` parameter also accepts three special options: `@hourly`, `@daily` and `@monthly`.


By default, schedule is setup in UTC timezone. You can set the schedule in a timezone using -t or --timezone option. The `--timezone` option supports only extended timezone formats like 'Asia/Tokyo', 'America/Los_Angeles' etc. Timezone abbreviations like PST, CST are *not* supported and might lead to unexpected schedules.

List the Schedules

You can see the list of currently scheduled entries by td connector:list.

...