Storing Logs via HTTP protocol

You can use HTTP to add logs from many places to td-agent.

Table of Contents

Prerequisites

  • Basic knowledge of Treasure Data (an understanding of the Getting Started guide).

Configuration

Please add the following lines to your /etc/td-agent/td-agent.conf file.

# HTTP input
<source>
  type http
  port 8888
</source>

# Treasure Data output
<match td.*.*>
  type tdlog
  endpoint api.treasuredata.com
  apikey ...
  auto_create_table
  buffer_type file
  buffer_path /var/log/td-agent/buffer/td
  use_ssl true
</match>

Uploading Data via HTTP

You can add logs via HTTP by sending JSON data. Please use the POST method and specify the database and table name by URL.

$ curl -X POST -d 'json={"action":"login","user":2}' \
  http://localhost:8888/td.testdb.www_access

td-agent continuously uploads logs every minute. Sending a SIGUSR1 signal will flush the buffer; upload will start immediately.

$ kill -USR1 `cat /var/run/td-agent/td-agent.pid`

To confirm that your data has been uploaded successfully, issue the td tables command as shown below.

$ td tables
+------------+------------+------+-----------+
| Database   | Table      | Type | Count     |
+------------+------------+------+-----------+
| testdb     | www_access | log  | 1         |
+------------+------------+------+-----------+

If you run into an issue, your log (/var/log/td-agent.log) is a good place to start your investigation.


Last modified: Aug 03 2015 00:01:48 UTC

If this article is incorrect or outdated, or omits critical information, please let us know. For all other issues, please see our support channels.