Data expiration is a mechanism that allows you to set an expiration interval (in days) for the data in a table. When an expiration period is set for a specific table, Treasure Data automatically filters out any records with a timestamp older than that of the expiration period and marks the records for purging from internal storage.
Use Case - Managing the Amount of Storage Consumed by a Table
For example, you can configure a 90-day data expiration policy to limit an events table to hold 90 days of history. Assuming relatively constant event traffic, this policy would in general limit the growth of the table without requiring ongoing management.
Non-Compliance with Data Retention Regulations. Data expiration in Treasure Data is not sufficient to comply with data privacy regulations and other industry-specific data retention regulations. In cases requiring immediate removal of data from TD, contact Treasure Data support for our recommended practices.
Configure Table Data Expiration
Use the TD Console
Navigate to Workbench > Databases.
Select a database.
Select the table.
Select ... > Table settings.
Select Expire Data?
Type the number of days you would like to keep the data active.
Use the TD Toolbelt
TD Toolbelt Example
$ td table:expire test_db test_table <expiration_days>
<expiration_days> is an integer number greater than 0.
Disabling Table Data Expiration
To disable the table data expiration, set the expiration days to 0. For example, using TD toolbelt:
$ td table:expire test_db test_table 0
Effects of Data Expiration
Review the following to understand how Treasure Data handles data expiration: