You can send Treasure Agent (td-agent) and Fluentd logs to the Treasure Data platform.

Capture td-agent / Fluentd logs

You can send and store td-agent logs in Treasure Data.

Prerequisites

Basic knowledge of Treasure Data Agent. The td-agent as a Server Side Agent article is a good place to start.
Basic knowledge of Treasure Data Agent / Fluentd configuration and syntax, see Configuration File Syntax, to understand the terms and concepts used in this article.

td-agent Logs

td-agent marks its own logs with the fluent tag. You can process td-agent logs by using <match fluent.**> (Of course, ** captures other logs) in <label @FLUENT_LOG>. If you define in <label @FLUENT_LOG> your configuration, then td-agent sends its own logs to this label. This is useful for monitoring td-agent logs.

For example, if you have the following configuration:

# omit other source / match
<label @FLUENT_LOG>
  <match fluent.*>
    @type stdout
  </match>
</label>

td-agent outputs fluent.info logs to stdout as follows:

2014-02-27 00:00:00 +0900 [info]: shutting down fluentd
2014-02-27 00:00:01 +0900 fluent.info: {"message":"shutting down fluentd"} # by <match fluent.*>
2014-02-27 00:00:01 +0900 [info]: process finished code = 0

Sending Logs to Treasure Data

To send td-agent logs to Treasure Data for storage you must modify the td-agent log configuration section. You need to include the modifications in the <match fluent.*> section as shown in the following example. Make sure to include your write-only API key in place of <YOUR TD API KEY>.

<label @FLUENT_LOG>
   <match fluent.*>
     @type tdlog
     apikey <YOUR TD API KEY>
     auto_create_table
     use_ssl true
     <buffer>
       @type file
       path /var/log/td-agent/buffer/td_logs
     </buffer>
   </match>
</label>

Save your changes to the configuration file and restart the td-agent.

Viewing Logs in Treasure Data

td-agent logs sent to Treasure Data are stored in a database named `fluent` on your Treasure Data instance. Within the fluent database, a series of tables are created corresponding to the different log levels used by the fluentd logger.

What’s Next?

For more information on td-agent's logs see Fluentd Logging.

Page tree

td-agent Logs Sent to Treasure Data