This is the record of all notable changes made to the Treasure Agent (td-agent) in the form of a change log. All notable changes to this project are documented in this file.
td-agent4 Newer More Stable
Learn about supported platforms and the differences between td-agent v2, v3, and v4.
Major features of td-agent v4:
Ruby 2.7
Fluentd v1
Arm64 Support
td-agent3 Legacy Stable
td-agent3 has moved into maintenance mode and will only receive an update if a security issue is found. Please use td-agent4 for a new deployment
This version uses Fluentd v1.
2020/06/23: Version v3.8.0
fluentd v1.11.1
td v0.16.9
ffi v1.13.1
tzinfo-data v1.2020.1
aws-sdk-core v3.100.0
aws-sdk-sqs v1.28.0
aws-sdk-s3 v1.69.1
fluent-plugin-s3 v1.3.2
fluent-plugin-webhdfs v1.2.5
ruby-kafka v1.1.0
rdkafka-ruby v0.8.0
fluent-plugin-rewrite-tag-filter v2.3.0
elasticsearch-ruby v6.8.2
fluent-plugin-elasticsearch v4.0.9
fluent-plugin-prometheus v1.8.0
winevt_c v0.8.1
fluent-plugin-windows-eventlog v0.7.0
fluent-plugin-parser-winevt_xml v0.2.2
Enhancements
rpm: Improve getent call in init.d script
rpm: Remove redhat-lsb-core dependency from CentOS 7
2020/04/24: Version v3.7.1
ruby 2.4.10
fluentd v1.10.2 (Include linger_timeout patch for Windows.)
async-http v0.50.8
tzinfo v2.0.2
aws-sdk-core v3.94.0
aws-sdk-s3 v1.63.0
fluent-plugin-s3 v1.3.1
fluent-plugin-windows-eventlog v0.5.4
2020/03/31: Version v3.7.0
fluentd v1.10.0
async-http v0.50.7
aws-sdk-core v3.92.0
aws-sdk-s3 v1.61.1
ruby-kafka v1.0.0
fluent-plugin-kafka v0.13.0
fluent-plugin-elasticsearch v4.0.7
fluent-plugin-prometheus_pushgateway v0.0.2
winevt_c v0.7.3
fluent-plugin-windows-eventlog v0.5.3
2020/02/18: Version v3.6.0
Amazon Linux 2: Support aarch64
fluentd v1.9.2
msgpack-ruby v1.3.3
cool.io v1.6.0
serverengine v2.2.1
tzinfo v2.0.1
td v0.16.8
fluent-plugin-td v1.1.0
aws-sdk-s3 v1.60.2
fluent-plugin-s3 v1.3.0
webhdfs v0.9.0
fluent-plugin-kafka v0.12.3
elasticsearch-ruby v6.8.1
fluent-plugin-elasticsearch v4.0.3
fluent-plugin-prometheus v1.7.3
fluent-plugin-record-modifier v2.1.0
winevt_c v0.7.0
fluent-plugin-windows-eventlog v0.5.0
fluent-plugin-systemd v1.0.2
fluent-plugin-prometheus_pushgateway v0.0.1
async-http v0.50.2
Enhancements
Support TD_AGENT_LOG_FILE envvar to modify log file location.
Fix useradd message bug in RPM's post script
2019/11/01: version v3.5.1
fluentd v1.7.4
ruby v2.4.9
td v0.16.5
tzinfo-data v1.2019.3
aws-sdk-s3 v1.52.0
fluent-plugin-s3 v1.2.0
rdkafka v0.7.0
fluent-plugin-kafka v0.12.1
fluent-plugin-elasticsearch v3.5.5
fluent-plugin-prometheus v1.7.0
winevt_c v0.6.1
fluent-plugin-windows-eventlog v0.4.3
td-agent.conf on Windows is not overwritten since the next version
See also https://github.com/treasure-data/omnibus-td-agent/pull/215
2019/08/27: version v3.5.0
fluentd v1.7.0
tzinfo v2.0.0
tzinfo-data v1.2019.2
aws-sdk-s3 v1.46.0
fluent-plugin-s3 v1.1.11
fluent-plugin-webhdfs v1.2.4
ruby-kafka v0.7.10
rdkafka v0.6.0
fluent-plugin-kafka v0.11.1
elasticsearch v6.8.0
fluent-plugin-elasticsearch v3.5.4
fluent-plugin-prometheus v1.5.0
fluent-plugin-windows-eventlog v0.3.0
Use /etc/default/td-agent as EnvironmentFile on Ubuntu/Debian
2019/05/18: version v3.4.1
ruby v2.4.6
fluent-plugin-s3 v1.1.10
fluent-plugin-rewrite-tag-filter v2.2.0
fluent-plugin-kafka v0.9.4
fluent-plugin-elasticsearch v3.5.1
fluent-plugin-prometheus v1.4.0
2019/05/04: Version v3.4.0
Fluentd v1.4.2
This includes several additional patches (TLS fix for Windows, tag parameters in in_forward, etc)
fluent-plugin-kafka v0.9.2
fluent-plugin-elasticsearch v3.4.3
fluent-plugin-s3 v1.1.9
serverengine v2.1.1
aws-sdk-core v3.48.3
aws-sdk-s3 v1.36.0
ruby-kafka v0.7.6
rdkafka 0.4.2 (only Linux)
tzinfo-data v1.2019.1
oj v3.7.11
Use /etc/sysconfig/td-agent as EnvironmentFile in systemd
2019/01/10: Version v3.3.0
ruby v2.4.5
Fluentd v1.3.3
fluent-plugin-kafka v0.8.3
fluent-plugin-elasticsearch v3.0.1
fluent-plugin-s3 v1.1.7
fluent-plugin-write-tag-filter v2.1.1
serverengine v2.1.0
This fixes the permission security issue of UNIX domain socket.
aws-sdk-core v3.44.2
aws-sdk-s3 v1.30.0
ruby-kafka v0.7.4
Drop kafka v0.10 support. If you want to connect to kafka v0.10, you need to downgrade ruby-kafka and fluent-plugin-kafka version.
tzinfo-data v1.2018.9
oj v3.7.4
td v0.16.4
2018/10/15: Version v3.2.1
Fluentd v1.2.6
fluent-plugin-kafka v0.7.9
fluent-plugin-elasticsearch v2.11.11
fluent-plugin-s3 v1.1.6
serverengine 2.0.7
ruby-kafka v0.6.8
2018/06/13: Version v3.2.0
Fluentd v1.2.2
fluent-plugin-kafka v0.7.3
fluent-plugin-elasticsearch v2.10.3
fluent-plugin-s3 v1.1.3
fluent-plugin-webhdfs v1.2.3
fluent-plugin-rewrite-tag-filter v2.1.0
Add fluent-plugin-record-modifier v1.1.0
postgresql 9.6.9
msgpack-ruby 1.2.4
td 0.16.1
bundler 1.16.2
yajl-ruby 1.4.0
tzinfo-data 1.2018.5
Support Ubuntu 18.04 64bit
2017/12/19: Version v3.1.1
Fluentd v1.0.2
fluent-plugin-kafka v0.6.5
fluent-plugin-elasticsearch v2.4.0
jemalloc 4.5.0
postgresql 9.6.6
msgpack-ruby 1.2.0
dig_rb 1.0.1
Add RuntimeDirectory to systemd unit
2017/12/1: Version v3.1.0
Ruby 2.4.2
Fluentd v0.14.25
fluent-plugin-s3 v1.1.0
fluent-plugin-kafka v0.6.4
fluent-plugin-webhdfs v1.2.2
fluent-plugin-td v1.0.0
fluent-plugin-rewrite-tag-filter v2.0.1
fluent-plugin-windows-eventlog v0.2.2
fluent-plugin-elasticsearch v2.3.0
bundler 1.6.0
msgpack-ruby 1.1.0
yajl-ruby 1.3.1
cool.io 1.5.3
oj 3.3.9
aws-sdk-s3 1.8.0
aws-sdk-sqs 1.3.0
td 0.15.7
2017/05/18: Version v3.0.1
Ruby 2.4.1
Fluentd v0.14.16
fluent-plugin-s3 v1.0.0.rc3
fluent-plugin-kafka v0.5.5
fluent-plugin-webhdfs v1.1.1
fluent-plugin-windows-eventlog v0.2.0
fluent-plugin-elasticsearch v1.9.5
msgpack-ruby 1.1.0
2017/01/08: Version v3.0.0
Ruby 2.4.0
Fluentd v0.14.11
fluent-plugin-s3 v1.0.0.rc1
fluent-plugin-kafka v0.5.0
fluent-plugin-td v1.0.0.rc1
fluent-plugin-elasticsearch v1.9.2
yajl-ruby v1.3.0
msgpack-ruby v1.0.2
Remove fluent-plugin-scribe and fluent-plugin-mongo from bundled plugins
Added Windows support
Dropped CentOS 5, Ubuntu Lucid, Ubuntu Precise, Debian Wheezy support
td-agent2 Unmaintained, Not Recommended for New Deployments
This version uses Fluentd v0.12 and the Ruby version is 2.5/2.1. This article describes the differences between td-agent2 and td-agent3. td-agent2 is for existing td-agent2 and fluentd v0.12 users. If you have new deployment, try td-agent3 version first.
2018/06/28: version v2.5.0
Fluentd v0.12.43
ruby 2.5.1
2017/01/01: Repository GPG Key Update
From 2017/01/01, the major Linux packaging system apt
has shut off the support for SHA1 encrypted GPG key for package signing. This is because SHA1 algorithm is considered weak.
Treasure Agent’s GPG key located at https://packages.treasuredata.com/GPG-KEY-td-agent
has been upgraded. To update your GPG key for better security:
If you’re using the old GPG key, you’ll see an error message similar to the following:
W: Signature by key XYZXYZXYZXYZXYZ uses weak digest algorithm (SHA1)
2017/10/04: version v2.3.6
fluentd v0.12.40
fluent-plugin-s3 v0.8.5
fluent-plugin-mongo v0.8.1
fluent-plugin-kafka v0.6.1
fluent-plugin-rewrite-tag-filter v1.5.6
fluentd-ui v0.4.4
cool.io v1.5.1
aws-sdk v2.10.45
ruby-kafka v0.4.2
rubygems v2.6.13
2017/04/26: version v2.3.5
fluentd v0.12.35
fluent-plugin-s3 v0.8.2
fluent-plugin-mongo v0.8.0
fluent-plugin-kafka v0.5.5
fluentd-ui v0.4.4
cool.io v1.4.6
yajl-ruby v1.3.0
aws-sdk v2.9.9
td-client v0.8.85
tzinfo v1.2.3
tzinfo-data v1.2017.2
ruby-kafka v0.3.17
mongo v2.2.7
fix logrotate script
2016/12/31: version v2.3.4
fluentd v0.12.31
fluent-plugin-s3 v0.8.0
fluent-plugin-mongo v0.7.16
fluent-plugin-kafka v0.4.1
fluentd-ui v0.4.4.rc1
td v0.15.0
oj v2.18.0
aws-sdk v2.6.42
td-client v0.8.85
tzinfo-data v1.2016.10
ruby-kafka v0.3.15
httpclient v2.8.2.4
Add 32bit package to Ubuntu 16.04
2016/09/30: version v2.3.3
fluentd v0.12.29
fluent-plugin-s3 v0.7.1
fluent-plugin-mongo v0.7.15
fluent-plugin-kafka v0.3.1
fluent-plugin-td v0.10.29
fluentd-ui v0.4.3
td v0.15.0
cool.io v1.4.5
aws-sdk v2.6.3
td-client v0.8.83
tzinfo-data v1.2016.6
ruby-kafka v0.3.15
jemalloc v4.2.1
2016/07/10: version v2.3.2
fluentd v0.12.26
fluent-plugin-s3 v0.6.8
fluent-plugin-webhdfs v0.4.2
fluent-plugin-mongo v0.7.13
fluent-plugin-rewrite-tag-filter v1.5.5
td v0.14.0
ruby v2.1.10
cool.io v1.4.4
sigdump v0.2.4
httpclient v2.7.2
oj v2.15.1
aws-sdk v2.3.14
webhdfs v0.8.0
tzinfo-data v1.2016.5
Ubuntu 16.04 64bit support
2016/02/11: version v2.3.1
fluentd v0.12.20
fluent-plugin-s3 v0.6.5
fluent-plugin-td v0.10.28
fluent-plugin-td-monitoring v0.2.2
fluent-plugin-mongo v0.7.12
fluent-plugin-rewrite-tag-filter v1.5.4
td v0.13.2
rubygems v2.4.8
httpclient v2.7.1
oj v2.14.4
aws-sdk v2.2.14
webhdfs v0.7.4
tzinfo-data v1.2016.1
2015/12/24: version v2.3.0
fluentd v0.12.19
fluent-plugin-s3 v0.6.4
fluent-plugin-td v0.10.28
fluent-plugin-mongo v0.7.11
fluent-plugin-rewrite-tag-filter v1.5.3
td v0.13.0
ruby v2.1.8
bundler v1.10.6
cool.io v1.4.2
oj v2.14.2
tzinfo-data v1.205.07
2015/06/29: version v2.2.1
fluentd v0.12.12
fluent-plugin-s3 v0.5.9
fluent-plugin-mongo v0.7.10
fluent-plugin-td v0.10.27
fluent-plugin-td-monitoring v0.2.1
td v0.11.10
fluentd-ui v0.4.2
rpm: Fix
-e
issue in init scriptdeb: Improve stop mechanizm to prevent killing other td-agent process in containers
deb: Add STOPTIMEOUT parameter to control shutdown td-agent process same as rpm
2015/04/06: version v2.2.0
fluentd v0.12.7
fluent-plugin-s3 v0.5.7
fluent-plugin-mongo v0.7.8
td v0.11.8.2
fluentd-ui v0.3.18
ruby 2.1.5
json 1.8.2
cool.io 1.3.0
Amazon Linux support
2015/03/26: version v2.1.5-1
Put logrotate setting in deb file package
Fix status command in rpm init script
Include PostgreSQL
2015/03/03: version v2.1.5
fluentd v0.10.61
fluent-plugin-s3 v0.5.6
fluent-plugin-mongo v0.7.6
fluent-plugin-td v0.10.26
td v0.11.8
td-client v0.8.68
2015/02/09: version v2.1.4
fluentd v0.10.60
fluent-plugin-s3 v0.5.3
fluent-plugin-mongo v0.7.5
fluent-plugin-td v0.10.25
fluentd-ui v0.3.14
msgpack 0.5.11
2014/12/23: version v2.1.3
fluentd v0.10.58
fluent-plugin-scribe v0.10.14
fluent-plugin-s3 v0.5.1
fluent-plugin-td v0.10.23
fluent-plugin-td-monitoring v0.2.0
fluentd-ui v0.3.11
RHEL/CentOS 7 support
2014/11/17: version v2.1.2
fluentd v0.10.57
fluent-plugin-scribe v0.1.3
fluent-plugin-mongo v0.7.4
fluent-plugin-s3 v0.4.3
fluent-plugin-webhdfs v0.4.1
fluent-plugin-td-monitoring v0.1.4
Remove fluent-plugin-flume
fluentd-ui v0.3.7
jemalloc 2.2.5
rubygems 2.2.1
Create /var/run/td-agent during post installation on rpm
2014/10/22: version v2.1.1
Ruby 2.1.3
fluentd v0.10.55
td v0.11.5
td-client v0.8.66
fluent-plugin-scribe v0.1.2
fluent-plugin-s3 v0.4.1
fluent-plugin-webhdfs v0.3.1
fluent-plugin-td v0.10.22
fluent-plugin-td-monitoring v0.1.3
Fix option handling in configtest
rpm: default timeout to 120 seconds
Change v1 non-quoted string literal behavior: https://groups.google.com/forum/#!topic/fluentd/4C36iAyI87E
2014/09/05: version v2.1.0
Ruby 2.1.2
fluentd v0.10.53
Use —use-v1-config by default
jemalloc v3.6.0
msgpack-ruby v0.5.8
Cool.io v1.2.4
td-agent1 Unmaintained and Not Recommended for New Deployments
2014/10/20: Version v1.1.21
fluentd v0.10.55
td v0.11.5
td-client v0.8.66
fluent-plugin-scribe v0.1.2
fluent-plugin-s3 v0.4.1
fluent-plugin-webhdfs v0.3.1
fluent-plugin-td v0.10.22
fluent-plugin-td-monitoring v0.1.3
Fix option handling in configtest
rpm: default timeout to 120 seconds
deb: fix cleanup routine in purge
2014/06/19: Version v1.1.20
fluentd v0.10.50
td-client v0.8.61
fluent-plugin-s3 v0.4.0
fluent-plugin-td v0.10.20
fluent-plugin-td-monitoring v0.1.2
rspec v3.0.0
Improve stop routine of init script in RPM
Set ‘ulimit -n 65536’ at start
2014/04/01: Version v1.1.19
fluentd v0.10.45
td-client v0.8.58
td v0.10.99
fluent-plugin-s3 v0.3.7
fluent-plugin-td v0.10.18
fluent-plugin-rewrite-tag-filter v1.4.1
fluent-plugin-mongo v0.7.3
fluent-plugin-td-monitoring v0.1.1
bundler to v1.3.6
webhdfs v0.5.5
aws-sdk-ruby v1.38.0
2013/12/05: Version v1.1.18
ruby v1.9.3-p484 (security fix)
fluentd v0.10.41
td-client v0.8.56
td v0.10.96
fluent-plugin-s3 v0.3.5
fluent-plugin-td v0.10.17
fluent-plugin-rewrite-tag-filter v1.3.1
2013/09/25: Version v1.1.17
fluentd v0.10.39
td-client v0.8.55
td v0.10.89
fluent-plugin-td v0.10.16
Fix configtest permission issue at restart
Create /etc/default/td-agent to set daemon options at Debian and Ubuntu
2013/08/30: Version v1.1.16
fluentd v0.10.38
td-client v0.8.54
td v0.10.86
Add configtest and use configtest at restart
2013/08/02: Version v1.1.15
fluentd v0.10.36
td-client v0.8.53
td v0.10.84
fluent-plugin-s3 v0.3.4
fluent-plugin-webhdfs v0.2.1
fluent-plugin-mongo v0.7.1
fluent-plugin-td v0.10.15
cool.io v1.1.1
2013/06/24: Version v1.1.14
fluentd v0.10.35
td-client v0.8.52
td v0.10.82
fluent-plugin-s3 v0.3.3
fluent-plugin-webhdfs v0.2.0
webhdfs v0.5.3
bson_ext v1.8.6
bson v1.8.6
mongo v1.8.6
yajl-ruby v1.1.0
json v1.7.7
2013/04/23: Version v1.1.13
fluent-plugin-td v0.10.14
td-client v0.8.48
td v0.10.76
2013/04/04: Version v1.1.12
fluentd v0.10.33
fluent-plugin-s3 v0.3.1
fluent-plugin-mongo v0.7.0
fluent-plugin-webhdfs v0.1.2
msgpack v0.4.7
bson_ext v1.8.4
bson v1.8.4
mongo v1.8.4
iobuffer v1.1.2
2012/12/07: Version v1.1.11
fluentd v0.10.30
fluent-plugin-s3 v0.2.5
fluent-plugin-td v0.10.13
fluent-plugin-mongo v0.6.11
2012/10/16: Version v1.1.10.3
td-client v0.8.34 to set the timeout for connect(2)
2012/10/15: Version v1.1.10.2
fluent-plugin-td v0.10.12
2012/10/15: Version v1.1.10.1
fixed /usr/bin/td regression of setting GEM_HOME and GEM_PATH
2012/10/15: Version v1.1.10
fluentd v0.10.27
fluent-plugin-mongo v0.6.9
fluent-plugin-webhdfs v0.5.1
fluent-plugin-td v0.10.11
enable debug agent by default (td-agent.conf)
set GEM_HOME and GEM_PATH at /usr/[s]bin/scripts to avoid RVM conflicts
2012/08/27: Version 1.1.9
fixed /usr/bin/td doesn’t take any command-line argument
Ubuntu LTS Precise 12.04 is now supported
allow overriding the variables by /etc/sysconfig/td-agent (rpm)
2012/07/23: Version 1.1.8
fluentd v0.10.25
fixed critical problem of duplicate daemon launch problem (both deb and rpm)
2012/06/12: Version 1.1.7
bson_ext v1.6.4
bson v1.6.4
mongo v1.6.4
fluent-plugin-td v0.10.7
td v0.10.25 (new)
install /usr/bin/td (new)
2012/05/20: Version 1.1.6
fluent-plugin-flume v0.1.1
remove ruby package dependency (rpm only)
2012/05/02: Version 1.1.5
ruby v1.9.3-p194 (security fix)
fluentd v0.10.22
add —with-libyaml-dir to ruby’s configure options
2012/04/23: Version 1.1.4.4
add td-libyaml to the dependency (rpm only)
support debian 5 lenny (deb only)
2012/04/17: Version 1.1.4
use ruby-1.9.3-p125
use jemalloc v2.2.5, to avoid memory fragmentations
fluentd v0.10.19
fluent-plugin-mongo v0.6.7
fluent-plugin-td v0.10.6
2012/03/24: Version 1.1.3.1
not to start daemon, at the first installation. (rpm only)
fix prelink Ruby breakage problem on CentOS 4 (rpm only)
fix ‘/etc/init.d/td-agent status’ problem on CentOS 4 (rpm only)
2012/03/10: Version 1.1.3
fluent-plugin-mongo v0.6.6
2012/02/22: Version 1.1.2.2
reduced # of dependent packages of RPM. The build time dependencies were unexpectedly included.
2012/02/21: Version 1.1.2.1
fixed problem of removing /etc/prelink.d/td-agent.conf file, when upgrading (rpm only)
2012/02/15: Version 1.1.2
fluentd v0.10.10
fluent-plugin-td v0.10.5
fluent-plugin-scribe v0.10.7
fluent-plugin-mongo v0.6.3
fluent-plugin-s3 v0.2.2
2012/01/24: Version 1.1.1
fluentd v0.10.9
fluent-plugin-scribe v0.10.6
fluent-plugin-mongo v0.6.2
fix /var/run/td-agent/ creation in init.d script
fix Ruby interpreter breakings by prelink, on 32-bit platform
2011/11/11: Version 1.1.0
fluentd v0.10.6
fluent-plugin-td v0.10.2
fluent-plugin-scribe v0.10.3
fluent-plugin-mongo v0.4.0
prevent prelink to break Ruby interpreter at the nightly job
2011/10/10: Version 1.0.11
fix gem installation order
2011/10/05: Version 1.0.10
fix posinst script for not overwriting user-installed config file
2011/10/01: Version 1.0.9
fluent v0.9.16
fluent-plugin-scribe v0.9.10
2011/09/20: Version 1.0.8
fluent v0.9.14
fluent-plugin-td v0.9.10