Incremental Loading for Data Extensions
Treasure Data supports incremental loading for Data Extensions that have a date field.
If incremental loading is selected, the integration loads records according to the range specified by the from_date and the fetch_days for the specified date field.
The following list provides details about all available options:
target: Target object. For example, a Customer or Invoice that accepts a search operation. (string, required)
email: Email address. (string, required)
password: Password. (string, required)
account: Account ID. (string, required)
role: Role ID. (string, default: nil)
sandbox: Use sandbox environment if true. (bool, default: false)
from_datetime: Fetch data after this time. (string, default: nil)
For example, you set up an incremental job with
Start Time = 2020-11-01T01:43:47.900Z and End Time = 2020-12-01T01:43:47.900Z
After the first run, the next start time will be calculated base on the precision setting.
HOURLY next start time will be 2020-12-01T02:43:47.900Z
DAILY next start time will be 2020-12-02T01:43:47.900Z
MONTHLY next start time will be 2021-01-01T01:43:47.900Z
And the End Time will be the current execution time.