Treasure Data supports incremental loading for Data Extensions that have a date field.
If incremental loading is selected, the integration loads records according to the range specified by the from_date and the fetch_days for the specified date field.
The following list provides details about all available options:
- target — Target object. For example, a Customer or Invoice that accepts a search operation. (string, required)
- email — Email address. (string, required)
- password — Password. (string, required)
- account — Account ID. (string, required)
- role — Role ID. (string, default: nil)
- sandbox — Use sandbox environment if true. (bool, default:false)
- from_datetime — Fetch data after this time. (string, default: nil)
For example, you set up an incremental job with
Start Time = 2020-11-01T01:43:47.900Z and End Time = 2020-12-01T01:43:47.900Z
After the first run, the next start time will be calculated base on the precision setting.
- HOURLY next start time will be 2020-12-01T02:43:47.900Z
- DAILY next start time will be 2020-12-02T01:43:47.900Z
- MONTHLY next start time will be 2021-01-01T01:43:47.900Z
And the End Time will be the current execution time.