Treasure Data provides a server-side agent called Treasure Agent (td-agent) to collect server-side logs and events. You can continuously import data using td-agent.

Prerequisites

• Basic knowledge of Treasure Data.

• Use td-agent 3, Fluentd v0.14 (v1.0) series.

For a quick hands-on experience, see Run a Query and Download Results.

What is Treasure Agent?

Treasure agent (td-agent) is a data collection daemon. It collects logs from various data sources and uploads them to Treasure Data.

Stream Data

Logs usually rotate on an hourly or daily basis based on time or size. This system quickly produces many large log files that need to be batch imported for further analysis. This is an outdated approach. Logs are better treated as continuously generated STREAMS as opposed to files.

Although bulk-import is supported, we recommend importing your data continuously via td-agent.