REST API (Import)

Treasure Data users can ingest data through the public REST API. This article describes the specification of the data import REST API.

Untitled-3
For REST API to control database / table / jobs / etc, please visit here. This article explains an API only for the data import.

Table of Contents

Prerequisites

  • Basic knowledge of Treasure Data

Endpoint

All API requests should be sent to in.treasuredata.com. Both POST and GET method are supported, but POST are recommended.

Limitation

We do not throttle access at the moment. However, we may limit access to our API as deemed necessary.

POST /postback/v3/event/{database}/{table}

Authentication

Every request must contain authentication information, provided by the special X-TD-Write-Key HTTP headers. The header format is as follows: The apikey can be retrieved from the console (click here). It’s recommended to use write-only API key.

": X-TD-Write-Key " + YourWriteOnlyApiKeyHere

Parameters

  • {database}: specify destination database name in URL. only alphabets/numbers or _ are allowed.
  • {table}: specify destination table name in URL. only alphabets/numbers or _ are allowed.

Request Body

Please specify JSON as a body. The JSON content will directly become a record in Treasure Data. Please also include Content-Type: application/json in HTTP request headers.

Response

  • HTTP 200 : Success
  • HTTP 400 : Bad Request
  • HTTP 403 : Forbidden

Example

Here’s the example curl command to import the record via POST method.

$ curl -D a -X POST \
  -H 'Content-Type: application/json' \
  -H 'X-TD-Write-Key: XXXX' \
  --data-binary '{"param1":"value1", "param2":1234}' \
  http://in.treasuredata.com/postback/v3/event/test_db/test_tbl
{}
$ cat a
HTTP/1.1 200 OK
Content-Type: application/json
Date: Thu, 03 Dec 2015 02:31:32 GMT
Content-Length: 2
Connection: keep-alive

This will ingest the records like below.

{
  'time': '1448344701',
  'param1': 'value1',
  'param2': 1234
}

GET /postback/v3/event/{database}/{table}

Authentication

Every request must contain authentication information, provided by the special td_write_key URL parameter. The The apikey can be retrieved from the console (click here). It’s recommended to use write-only API key.

http://in.treasuredata.com/postback/v3/event/{database}/{table}?td_write_key=XXXYYYZZZZ&...

Parameters

Please specify these two parameters with a path.

  • {database}: specify destination database name in URL. only alphabets/numbers or _ are allowed.
  • {table}: specify destination table name in URL. only alphabets/numbers or _ are allowed.

Any URL parameters, except special parametes listed below, is considered as a column inside the record.

  • td_write_key: Already described above.
  • td_record_time: Please specify UNIX timestamp value, to set time column in Treasure Data. If it’s not specified, the time records were received will be used instead.
  • td_global_id: If we add td_global_id=td_global_id in the query string of postback url, API add a column named td_global_id with value of actual td_global_id into the record.
  • td_ip: If we add td_ip=td_ip in the query string of postback url, API add a column named td_ip with value of source IP into the record.
  • td_ua: If we add td_ua=td_ua in the query string of postback url, API add a column named td_ua with value of actual user-agent into the record.

All parameter’s value is considered as string, and you don’t have a way to change it at the ingestion time. That’s why we’re recommending POST, where you can specify record as JSON with types.

Response

  • HTTP 200 : Success
  • HTTP 400 : Bad Request
  • HTTP 403 : Forbidden

Example

Here’s the example curl command to import the record via GET method.

$ curl -D a \
  "http://in.treasuredata.com/postback/v3/event/test_db/test_tbl?td_write_key=XXXYYYZZZ&param1=value1&param2=value2"
{}
$ cat a
HTTP/1.1 200 OK
Content-Type: application/json
Date: Sat, 09 Jan 2016 01:08:21 GMT
Content-Length: 2
Connection: keep-alive

This will ingest the records like below.

{
  'time': '1448344701',
  'param1': 'value1',
  'param2': 'value2'
}

Last modified: Dec 24 2016 01:51:48 UTC

If this article is incorrect or outdated, or omits critical information, please let us know. For all other issues, please see our support channels.