Scheduled execution supports additional configuration parameters that control the behavior of the data connector during its periodic attempts to fetch data from the integration source:
incrementalThis configuration is used to control the load mode, which governs how the data connector fetches data from the integration based on one of the native timestamp or numeric field associated with each object
incremental_columnnThis configuration is used to define a based column to imported into Treasure Data. You can define only one column for this field. Suggested value is created, createdTimestamp, updated, updatedTimestamp
Timezone abbreviations like PST, CST are not supported and may lead to unexpected schedules.
The `--timezone` option supports only extended timezone formats like 'Asia/Tokyo', 'America/Los_Angeles' etc.
Here’s an example of a load file using incremental mode
Create the Schedule
A new schedule can be created using the td connector:create command. The name of the schedule, cron-style schedule, the database and table where their data will be stored, and the data connector configuration file are required.
The `cron` parameter accepts these options: `@hourly`, `@daily` and `@monthly`.
By default, the schedule is setup in UTC timezone. You can set the schedule in a timezone using -t or --timezone option.
For example, you can create a scheduled job using the command td connector:create to run daily: