Skip to content
Last updated

Achieving Time Partitioning In S3 Of Data Exported Using Bulk Export

A common question we've received is "Can Treasure Data make sure the data exported through this process is partitioned according to hourly bucket, similar to the partitioning strategy that is maintained within the core Treasure Data system?"

The Bulk Export command no longer supports partitioning of exported data. This is to optimize the speed of export, which was too slow to meet requirements.

If you do require partitioning, we recommend using this command to export 1 hour segments at a time – automating the process with a script. While we know this isn’t the most convenient approach, use this approach to achieve any time-based partitioning in your bulk exported data.