Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Data expiration is a mechanism that allows you to set an expiration interval (in days) for the data in a table. You can use the TD Console to configure table data expiration or Hive and Presto query engines. When an expiration period is set for a specific table, Treasure Data automatically filters out any records with a timestamp older than that of the expiration period and marks the records for purging from internal storage.

Children Display

Data Expiration in the TD Console

Use Case - Managing the Amount of Storage Consumed by a Table

For example, you can configure a 90-day data expiration policy to limit an events table to hold 90 days of history. Assuming relatively constant event traffic, this policy would in general limit the growth of the table without requiring ongoing management.

Non-Compliance with Data Retention Regulations. Data expiration in Treasure Data is not sufficient to comply with data privacy regulations and other industry-specific data retention regulations. In cases requiring immediate removal of data from TD, contact Treasure Data support for our recommended practices.

Configure Table Data Expiration

...

in the TD Console

  1. Navigate to Workbench > Databases.

  2. Select a database.

  3. Select the table.

  4. Select ... > Table settings.

  5. Select Expire Data?

  6. Type the number of days you would like to keep the data active.

  7. Select Submit.

Data Expiration in the TD Toolbelt

Use the TD Toolbelt

TD Toolbelt Example

...

Code Block
linenumberstrue
$ td table:expire test_db test_table 0

Data Expiration in Hive and Presto Query Engines


Effects of Data Expiration

...