Skip to content
Products
/
/
/
Fluentd Logs Sent to Trea...
Last updated

Fluentd Logs Sent to Treasure Data

td-agent was discontinued in December 2023 and has been replaced by fluent-package. The fluent-package is the official successor maintained by the Cloud Native Computing Foundation.

You can send Fluentd logs to the Treasure Data platform.

Capture Fluentd logs

You can send and store Fluentd logs in Treasure Data.

Prerequisites

  • Basic knowledge of Fluentd.
  • Basic knowledge of Fluentd configuration and syntax, see Configuration File Syntax, to understand the terms and concepts used in this article.

Fluentd Logs

Fluentd marks its own logs with the fluent tag. You can process Fluentd logs by using <match fluent.**> (Of course, ** captures other logs) in <label @FLUENT_LOG>. If you define in <label @FLUENT_LOG> your configuration, then Fluentd sends its own logs to this label. This is useful for monitoring Fluentd logs.

For example, if you have the following configuration:

# omit other source / match
<label @FLUENT_LOG>
  <match fluent.*>
    @type stdout
  </match>
</label>

Fluentd outputs fluent.info logs to stdout as follows:

2014-02-27 00:00:00 +0900 [info]: shutting down fluentd
2014-02-27 00:00:01 +0900 fluent.info: {"message":"shutting down fluentd"} # by <match fluent.*>
2014-02-27 00:00:01 +0900 [info]: process finished code = 0

Sending Logs to Treasure Data

To send Fluentd logs to Treasure Data for storage, you must modify the Fluentd log configuration section. You need to include the modifications in the <match fluent.*> section, as shown in the following example. Ensure to include your write-only API key instead of <YOUR TD API KEY>.

<label @FLUENT_LOG>
  <match fluent.*>
    @type tdlog
    apikey <YOUR TD API KEY>
    auto_create_table
    use_ssl true
    <buffer>
      @type file
      path /var/log/fluent/buffer/td_logs
    </buffer>
  </match>
</label>

Save your changes to the configuration file and restart Fluentd.

Viewing Logs in Treasure Data

Fluentd logs sent to Treasure Data are stored in a database named fluent on your instance. Within the fluent database, a series of tables are created corresponding to the different log levels used by the fluentd logger.

What's Next?

For more information on Fluentd's logs see Fluentd Logging.

Previous page
Next page