# Monitoring TD Agent

You can monitor the Treasure Agent (`td-agent`) daemon to collect metrics or to troubleshoot your system.

* [Prerequisites](#prerequisites)
* [Fluentd Metrics Monitoring](#fluentd-metrics-monitoring)
* [Monitoring via HTTP](#monitoring-via-http)
* [Monitoring with Prometheus or Datadog](#monitoring-with-prometheus-or-datadog)
* [Process Monitoring](#process-monitoring)
* [Port Monitoring](#port-monitoring)
* [Debug Port](#debug-port)


# Prerequisites

* Basic knowledge of Treasure Data.
* Basic knowledge of td-agent.


# Fluentd Metrics Monitoring

## Monitoring via HTTP

`td-agent` has a built-in monitoring agent to retrieve internal metrics in JSON via HTTP. Add the following lines to your configuration file.


```conf
<source>
  type monitor_agent
  bind 0.0.0.0
  port 24220
</source>
```

`disable_node_info` (default `true`): Send system metrics, CPU / Memory / Disk, or not.

Restart the agent and get the metrics via HTTP.


```bash
$ curl http://host:24220/api/plugins.json
{"plugins":[{"plugin_id":"object:3fec669d6ac4","type":"forward","output_plugin":false,"config":{"type":"forward"}},{"plugin_id":"object:3fec669daf98","type":"http","output_plugin":false,"config":{"type":"http","port":"8888"}},{"plugin_id":"object:3fec669dfa48","type":"monitor_agent","output_plugin":false,"config":{"type":"monitor_agent","port":"24220"}},{"plugin_id":"object:3fec66a52e94","type":"debug_agent","output_plugin":false,"config":{"type":"debug_agent","port":"24230"}},{"plugin_id":"object:3fec66ae3dcc","type":"stdout","output_plugin":true,"config":{"type":"stdout"}},{"plugin_id":"object:3fec66aead48","type":"forward","output_plugin":true,"buffer_queue_length":0,"buffer_total_queued_size":0,"retry_count":0,"config":{"type":"forward","host":"192.168.0.11"}}]}%
```

## Monitoring with Prometheus or Datadog

Additionally, td-agent works with monitoring tools such as Prometheus, Datadog, etc. Our recommendation is to use Prometheus because Treasure Data collaborates with the CNCF (Cloud Native Computing Foundation).

* [Monitoring Fluentd (Prometheus)](https://docs.fluentd.org/monitoring-fluentd/monitoring-prometheus)
* [Monitoring Fluentd (Datadog)](https://docs.datadoghq.com/integrations/fluentd/)


# Process Monitoring

Two `ruby` processes (parent and child) are executed. Ensure that these processes are running.


```bash
/usr/lib/fluent/ruby/bin/ruby /usr/sbin/td-agent \
  --daemon /var/run/td-agent/td-agent.pid \
  --log /var/log/td-agent/td-agent.log
```

For td-agent on Linux, you can check the process status with the following command. You see the status of two processes if there are no issues.


```bash
$ ps w -C ruby -C td-agent --no-heading
32342 ?        Sl     0:00 /usr/lib/fluent/ruby/bin/ruby /usr/sbin/td-agent --daemon /var/run/td-agent/td-agent.pid --log /var/log/td-agent/td-agent.log
32345 ?        Sl     0:01 /usr/lib/fluent/ruby/bin/ruby /usr/sbin/td-agent --daemon /var/run/td-agent/td-agent.pid --log /var/log/td-agent/td-agent.log
```

# Port Monitoring

td-agent opens the following ports by default. We recommend checking the availability of these ports.

* TCP 0.0.0.0 8888 (HTTP)
* TCP 0.0.0.0 24224 (Forward)


If you don’t send any data, the daemon doesn’t do anything.

## Debug Port

A debug port for local communication is also opened.

* TCP 127.0.0.1 24230