You can send Treasure Agent (td-agent) and Fluentd logs to the Treasure Data platform.
Capture td-agent / Fluentd logs
You can send and store td-agent logs in Treasure Data.
Prerequisites
Basic knowledge of Treasure Data Agent. The td-agent as a Server Side Agent article is a good place to start.
Basic knowledge of Treasure Data Agent / Fluentd configuration and syntax, see Configuration File Syntax, to understand the terms and concepts used in this article.
td-agent Logs
td-agent marks its own logs with the fluent
tag. You can process td-agent logs by using <match fluent.**>
(Of course, **
captures other logs) in <label @FLUENT_LOG>
. If you define in <label @FLUENT_LOG>
your configuration, then td-agent sends its own logs to this label. This is useful for monitoring td-agent logs.
For example, if you have the following configuration:
# omit other source / match <label @FLUENT_LOG> <match fluent.*> @type stdout </match> </label>
td-agent outputs fluent.info
logs to stdout as follows:
2014-02-27 00:00:00 +0900 [info]: shutting down fluentd 2014-02-27 00:00:01 +0900 fluent.info: {"message":"shutting down fluentd"} # by <match fluent.*> 2014-02-27 00:00:01 +0900 [info]: process finished code = 0
Sending Logs to Treasure Data
To send td-agent logs to Treasure Data for storage you must modify the td-agent log configuration section. You need to include the modifications in the <match fluent.*> section as shown in the following example. Make sure to include your write-only API key in place of <YOUR TD API KEY>.
<label @FLUENT_LOG> <match fluent.*> @type tdlog apikey <YOUR TD API KEY> auto_create_table use_ssl true <buffer> @type file path /var/log/td-agent/buffer/td_logs </buffer> </match> </label>
Save your changes to the configuration file and restart the td-agent.
Viewing Logs in Treasure Data
td-agent logs sent to Treasure Data are stored in a database named `fluent` on your Treasure Data instance. Within the fluent database, a series of tables are created corresponding to the different log levels used by the fluentd logger.
What’s Next?
For more information on td-agent's logs see Fluentd Logging.