Treasure Data provides td-agent to collect server-side logs and events, to stream data from .NET applications.
Basic knowledge of .NET
Basic knowledge of Treasure Data
td-agenton your application servers. td-agent sits within your application servers, focusing on uploading application logs to the cloud.
td-agent receives the records via TCP/HTTP, buffers the records, and uploads the data to the cloud every 5 minutes. Because the daemon runs on a local node, the logging latency is negligible.
td-agent Install Options
td-agent, run one of the following commands based on your environment. The agent program is installed automatically by using the package management software for each platform like rpm/deb/dmg.
Ubuntu and Debian
Legacy support for EOL versions is still available
You can choose Amazon Linux 1 or Amazon Linux 2. Refer to Installing td-agent on AWS Linux.
MacOS X 10.11+
MacOS X 10.11.1 (El Capitan) introduced some security changes. After the td-agent is installed, edit the /Library/LaunchDaemons/td-agent.plist file to change /usr/sbin/td-agent to /opt/td-agent/usr/sbin/td-agent.
Windows Server 2012+
The Windows installation requires the steps detailed in:
Opscode Chef Repository
You can read more about the repository.
AWS Elastic Beanstalk is also supported. Windows is not supported.
Specify your API key by setting the
apikey option in your
YOUR_API_KEY should be your actual API key string. You can retrieve your API key from your profile in TD Console. Using a [write-only API key](access-control#rest-apis-access) key is recommended.
Restart your agent after the following lines are added:
td-agent accepts data via port 24224, buffers the data (var/log/td-agent/buffer/td), and automatically uploads the data into the cloud.
Post the Records via HTTP
Use the following code snippets to post the records to the local Treasure Agent via HTTP.
Confirming Data Import
Execute your program.
Sending a SIGUSR1 signal will flush td-agent’s buffer. The upload starts immediately.
From TD Console
To confirm that your data has been uploaded successfully, check your dataset in the TD Console, Databases view.
Or, issue the td tables command if you have a CLI client.
High-Availability Configurations of td-agent
For high-traffic websites (more than 5 application nodes), use a high availability configuration of td-agent to improve data transfer reliability and query performance.
Monitoring td-agent itself is also important. For general monitoring methods for td-agent, see Monitoring td-agent.
td-agent is fully open-sourced under the Fluentd project.