Skip to content
Products
/
/
/
Monitoring Fluentd
Last updated

Monitoring Fluentd

You can monitor the Fluentd daemon to collect metrics or to troubleshoot your system.

td-agent was discontinued in December 2023 and has been replaced by fluent-package. The fluent-package is the official successor maintained by the Cloud Native Computing Foundation.

Prerequisites

  • Basic knowledge of Treasure Data.
  • Basic knowledge of Fluentd.

Fluentd Metrics Monitoring

Monitoring via HTTP

Fluentd has a built-in monitoring agent to retrieve internal metrics in JSON via HTTP. Add the following lines to your configuration file.

<source>
  @type monitor_agent
  bind 0.0.0.0
  port 24220
</source>

disable_node_info (default true): Send system metrics, CPU / Memory / Disk, or not.

Restart the agent and get the metrics via HTTP.

$ curl http://host:24220/api/plugins.json
{"plugins":[{"plugin_id":"object:3fec669d6ac4","type":"forward","output_plugin":false,"config":{"type":"forward"}},{"plugin_id":"object:3fec669daf98","type":"http","output_plugin":false,"config":{"type":"http","port":"8888"}}]}

Monitoring with Prometheus or Datadog

Additionally, Fluentd works with monitoring tools such as Prometheus, Datadog, etc. Our recommendation is to use Prometheus because Treasure Data collaborates with the CNCF (Cloud Native Computing Foundation).

Process Monitoring

Two ruby processes (parent and child) are executed. Ensure that these processes are running.

/opt/fluent/bin/ruby /opt/fluent/bin/fluentd \
  --config /etc/fluent/fluentd.conf \
  --log /var/log/fluent/fluentd.log

For Fluentd on Linux, you can check the process status with the following command. You see the status of two processes if there are no issues.

$ ps w -C ruby -C fluentd --no-heading
32342 ?        Sl     0:00 /opt/fluent/bin/ruby /opt/fluent/bin/fluentd --config /etc/fluent/fluentd.conf
32345 ?        Sl     0:01 /opt/fluent/bin/ruby /opt/fluent/bin/fluentd --config /etc/fluent/fluentd.conf

Port Monitoring

Fluentd opens the following ports by default. We recommend checking the availability of these ports.

  • TCP 0.0.0.0 8888 (HTTP)
  • TCP 0.0.0.0 24224 (Forward)

If you don't send any data, the daemon doesn't do anything.

Debug Port

A debug port for local communication is also opened.

  • TCP 127.0.0.1 24230

Other Resources

Previous page
Next page