Visit our new documentation site! This documentation page is no longer updated.

One-Time Import

This article explains how to import data using the td table:import command.

Table of Contents

Prerequisites

Why One-Time Import?

If you want to import a small amount of data (perhaps only a few small files), one-time import is a quick and simple option.

This method doesn’t use MessagePack compression and doesn’t retry if there is a network failure during the transfer. If a more reliable method is desired, please consider using Bulk Import instead.

The td table:import Command

To perform one-time import, please use the td table:import command. This command takes the file path and data format as arguments and uploads the data into the cloud. It currently supports the following data formats.

JSON

In order to perform one-time import with the JSON data format option, the files should be formatted as follows:

  • One JSON-map per line.
  • A time field is required. This field indicates the time in which the event was generated.

For example, entries could be formatted as follows:

{"action":"login","user":2,"time":"2011-08-02 03:06:32 +0900"}
{"action":"login","user":4,"time":"2011-08-02 03:06:32 +0900"}
{"action":"login","user":0,"time":"2011-08-02 03:06:32 +0900"}
...

You can import your data into the test_table table within your test_db database as follows.

$ td table:import test_db test_table \
  --format json \
  --time-key time \
  file1.json file2.json

Apache Log

The td table:import command can also parse Apache Logs stored in the ‘combined’ format. The command automatically splits the log records into meaningful fields (ex: time, user-agent, etc.) and imports the data into the cloud.

$ td table:import test_db test_table --format apache access_log.txt

Syslog

The td table:import command can also parse logs stored in the ‘syslog’ format.

$ td table:import test_db test_table --format syslog syslog.txt

Last modified: Feb 24 2017 09:27:52 UTC

If this article is incorrect or outdated, or omits critical information, let us know. For all other issues, access our support channels.