You can collect data from server-side applications, games, and IoT devices that are based in C++. This article describes how to reliably import data from your C and C++ applications, using Fluent Bit.


Prerequisites


  • Basic knowledge of C/C++ Application Development.

  • Basic knowledge of Treasure Data.

About Fluent Bit

Fluent Bit is a specialized open-source data collector tool for embedded operating systems and is developed by Treasure Data. Fluent Bit can be used as a standalone tool and can also be embedded in your C or C++ applications.

By embedding Fluent Bit in your C/C++ applications, you can have robust and light-weight data collection mechanism (buffering, retry, etc) inside your applications.

Embeding Fluent Bit and Connecting to Treasure Data

Get your Treasure Data API Key

Retrieve your Write-Only API key from here.

Prepare Configuration File

Prepare your configuration file by specifying: your TD Write-Only API key, the destination database, and table name. An example configuration is available here.

# Treasure Data Output
# ===================
# This configuration file specifies the information to be used
# when flushing information to TreasureData.com service. All
# key fields in the 'TD' section are mandatory.
[TD]
    # API
    # ===
    # The TreasureData API key. To obtain the TD API Key log into your
    # Treasure Data Console:
    #
    # 1. Go to https://console.treasuredata.com/users/current
    # 2. API Keys box: copy the API key hash
    API YOUR_WRITE_ONLY_API_KEY

    # Database
    # ========
    # Specify the name of your database. The database must exist.
    Database db_example

    # Table
    # =====
    # Specify the database table name where the records will be stored
    Table    table_example

Embed fluent-bit into your Applications

Retrieve the entire source code from GitHub.

When Fluent Bit is built, it creates the final binary and a shared library version called libfluent-bit.so which can be linked in your projects. The library allows you to enqueue data into the engine and return quickly. All the hard work is done in one posix thread that works in an asynchronous mode. Your caller process or thread will not be blocked.

Here’s an example program to send the data to Treasure Data.

#include <unistd.h>
#include <fluent-bit.h>

int main(int argc, char **argv)
{
    int i;
    int n;
    int ret;
    char tmp[256];
    struct flb_config *config;

    if (argc < 2) {
        fprintf(stderr, "Usage: td /path/to/configuration.file\n");
        exit(EXIT_FAILURE);
    }

    /* Create configuration context */
    config = flb_config_init();
    if (!config) {
        exit(EXIT_FAILURE);
    }

    /* Enable verbose messages */
    flb_config_verbose(FLB_TRUE);

    /* Load a configuration file (required by TD output plugin) */
    ret = flb_lib_config_file(config, argv[1]);

    /* Initialize library */
    ret = flb_lib_init(config, (char *) "td");
    if (ret != 0) {
        exit(EXIT_FAILURE);
    }

    /* Start the background worker */
    flb_lib_start(config);

    /* Push some data */
    for (i = 0; i < 100; i++) {
        n = snprintf(tmp, sizeof(tmp) - 1, "{\"key\": \"val %i\"}", i);
        flb_lib_push(config, tmp, n);
    }

    flb_lib_stop(config);
    return 0;
}

You only need to include the fluent-bit.h header and use the functions in the right order. The description of each function is available here:

Further Information

More information is available in the following links:

  • No labels