# Rails Apps Import Integration

td-agent was discontinued in December 2023 and has been replaced by **fluent-package**. The fluent-package is the official successor maintained by the [Cloud Native Computing Foundation](https://www.cncf.io/projects/).

Treasure Data provides [Fluentd](/products/customer-data-platform/integration-hub/streaming/td-agent/about-treasure-data-s-server-side-agent) to collect server-side logs and events and to seamlessly import the data from Ruby on Rails applications.

## Prerequisites

- Basic knowledge of Ruby, Rails, Gems, and Bundler.
- Basic knowledge of Treasure Data.
- Ruby 3.3 or higher (for local testing).


## Installing Fluentd

Install Fluentd (fluent-package) on your application servers. Fluentd sits within your application servers, focusing on uploading application logs to the cloud.

![](/assets/image-20191007-035026.6a881b783b2ff61b88f50c884f82ed90da06d32bddf4462fec948d0e7710f572.dffa02f7.png)

The [td-logger-ruby](http://github.com/treasure-data/td-logger-ruby) library enables Ruby on Rails applications to post records to their local Fluentd. Fluentd, in turn, receives the records, buffers them, and uploads the data to the cloud every 5 minutes. Because the daemon runs on a local node, the logging latency is negligible.

## Fluentd (fluent-package) Install Options

td-agent was discontinued in December 2023 and has been replaced by **fluent-package**. The fluent-package is the official successor maintained by the [Cloud Native Computing Foundation](https://www.cncf.io/projects/). For migration guidance from td-agent, see [Fluentd Installation Guide](https://docs.fluentd.org/installation/install-fluent-package).

To install `fluent-package`, run one of the following commands based on your environment.

### RHEL/CentOS/Rocky Linux


```bash
# fluent-package 6 LTS (recommended)
curl -fsSL https://fluentd.cdn.cncf.io/sh/install-redhat-fluent-package6-lts.sh | sh
```

### Ubuntu


```bash
# Ubuntu 24.04 Noble - fluent-package 6 LTS
curl -fsSL https://fluentd.cdn.cncf.io/sh/install-ubuntu-noble-fluent-package6-lts.sh | sh

# Ubuntu 22.04 Jammy - fluent-package 6 LTS
curl -fsSL https://fluentd.cdn.cncf.io/sh/install-ubuntu-jammy-fluent-package6-lts.sh | sh
```

### Debian


```bash
# Debian Bookworm - fluent-package 6 LTS
curl -fsSL https://fluentd.cdn.cncf.io/sh/install-debian-bookworm-fluent-package6-lts.sh | sh
```

### Amazon Linux


```bash
# Amazon Linux 2023 - fluent-package 6 LTS
curl -fsSL https://fluentd.cdn.cncf.io/sh/install-amazon2023-fluent-package6-lts.sh | sh
```

### Windows

Download the MSI installer from:

- [fluent-package 6 LTS for Windows](https://fluentd.cdn.cncf.io/lts/6/windows/index.html)


After installation:

1. Edit the configuration file at `C:/opt/fluent/etc/fluent/fluentd.conf`
2. Start the service using `net start fluentdwinsvc` or via Services administrative tool


### macOS

fluent-package for macOS is planned to be available via Homebrew. For current installation options, see [Fluentd Installation Guide](https://docs.fluentd.org/installation).

## Starting the Service

After installation, start and verify the Fluentd service.

### Linux


```bash
sudo systemctl start fluentd.service
sudo systemctl status fluentd.service
```

The configuration file is located at `/etc/fluent/fluentd.conf`.

### Windows


```cmd
net start fluentdwinsvc
```

The configuration file is located at `C:\opt\fluent\etc\fluent\fluentd.conf`.

### macOS (gem installation)


```bash
fluentd -c /path/to/fluentd.conf
```

For more details, see the [Fluentd Documentation](https://docs.fluentd.org/).

## Modifying fluentd.conf

Specify your API key by setting the `apikey` option in your `/etc/fluent/fluentd.conf` file (for fluent-package).


```conf
# Input from Logging Libraries
<source>
  @type forward
  port 24224
</source>

# Treasure Data Output
<match td.*.*>
  @type tdlog
  endpoint api.treasuredata.com
  apikey YOUR_API_KEY
  auto_create_table
  use_ssl true
  <buffer>
    @type file
    path /var/log/fluent/buffer/td
  </buffer>
</match>
```

`YOUR_API_KEY` should be your actual apikey string. You can retrieve your API key from your profiles in TD Console. Using a [write-only API key](/products/my-settings/getting-your-api-keys) is recommended.

Restart the Fluentd service after adding the following lines:


```bash
# Linux
sudo systemctl restart fluentd.service

# macOS (gem installation)
# Restart the Fluentd process manually
```

Fluentd accepts data via port 24224, buffers the data (`/var/log/fluent/buffer/td`), and automatically uploads the data into the cloud.

## Using td-logger-ruby

Add the ‘td’ gem to your Gemfile.


```
gem 'td', "~> 0.10.6"
```

Modify the *config/treasure_data.yml* file as follows:


```
production:
  agent: "localhost:24224"
  tag: td.production_database_name
  debug_mode: false

development:
  agent: "localhost:24224"
  tag: td.development_database_name
  debug_mode: false

test:
```

Insert the appropriate logging code into your application.


```
# Example1: login event
TD.event.post('login', {:uid=>123})

# Example2: follow event
TD.event.post('follow', {:uid=>123, :from=>'TD', :to=>'Heroku'})

# Example3: pay event
TD.event.post('pay',
              {:uid=>123, :item_name=>'Stone of Jordan',
               :category=>'ring', :price=>100, :count=>1})
```

## Confirming Data Import

Execute the program.


```bash
$ ruby test.rb
```

Sending a SIGUSR1 signal flushes Fluentd's buffer. The upload starts immediately.


```bash
# Linux
$ kill -USR1 $(cat /var/run/fluent/fluentd.pid)

# macOS (gem installation)
# Send SIGUSR1 to the Fluentd process
```

### From TD Console

To confirm that your data has been uploaded successfully, check your data set.

### From CLI

Or, use the *td tables* command if you have a CLI.


```
$ td tables
+------------+------------+------+-----------+
| Database   | Table      | Type | Count     |
+------------+------------+------+-----------+
| test_db    | login      | log  | 1         |
| test_db    | follow     | log  | 1         |
| test_db    | pay        | log  | 1         |
+------------+------------+------+-----------+
```

## Production Deployments

### Use Rack-Based Server Deployments

We recommend that you use *unicorn*, *thin*, *mongrel*, etc. Other setups have not been fully validated.

### High Availability Configurations of Fluentd

For high-traffic websites (more than 5 application nodes), use a high availability configuration of Fluentd to improve data transfer reliability and query performance.

- [High-Availability Configurations of Fluentd](/products/customer-data-platform/integration-hub/streaming/td-agent/configuring-td-agent-for-high-availability)


### Monitoring Fluentd

Monitoring Fluentd itself is also important. For general monitoring methods for Fluentd, see [Monitoring Fluentd](/products/customer-data-platform/integration-hub/streaming/td-agent/monitoring-td-agent).

Fluentd is fully open-sourced under the [Fluentd project](http://fluentd.org/).