The following operations allow you to import data into the tables you have set up for your queries and workflows. Import requests should serialize objects using gzipped MessagePack format. This is normally done by using fluentd or using Treasure Data or fluentd libraries but can be done manually. In words, the client should, with the objects to be imported:

  • Serialize objects to MessagePack

  • Concatenate objects together (objects are concatenated with no separator, the body is not an array)

  • Compress the payload with gzip.

This is usually done in a streaming manner. For additional information, see an example of this implemented in Java.

Description

API call (or REST call)

Imports data

table/import/:database_name/:table_name/:data_format

Imports data and specifies a unique ID for your import batch

table/import/:database_name/:table_name/:data_format

PUT /table/import/:database_name/:table_name/:data_format

This operation imports data.

Verb

URI

PUT

table/import/:database_name/:table_name/:data_format

 URI parameters

Parameter Name

Required

Type

Example

database_name

Y

String

"Alpha"

table_name

Y

String

"Bravo"

data_format

Y

String

"msgpack.gz" (required value)

Sample Request

<data in gzipped msgpack format>

Sample Response

{
  "elapsed_time": 0,
  "database": "Alpha",
  "md5_hex": "2ac30b788fd813e9ac7a3af9d395e109",
  "table": "Bravo",
  "unique_id": "2ac30b788fd813e9ac7a3af9d395e109"
}

PUT /table/import_with_id/:database_name/:table_name/:unique_id/:data_format

This operation allows you to import data with a unique ID, which can be used to deduplicate data.

Verb

URI

PUT

/table/import_with_id/:database_name/:table_name/:unique_id/:data_format

URI Parameters

Parameter Name

Required

Type

Example

database_name

Y

String

"Alpha"

table_name

Y

String

"Bravo"

unique_id

Y

String

(HexValue) [0-9a-f]+

"2ac30b788fd813e9ac7a3af9d395e109"

data_format

Y

String

"msgpack.gz"

Sample Response

{
  "elapsed_time": 0,
  "database": "Alpha",
  "md5_hex": "2ac30b788fd813e9ac7a3af9d395e109",
  "table": "Bravo",
  "unique_id": "2ac30b788fd813e9ac7a3af9d395e109"
}

 

  • No labels