The samples shown on this page let you access your data through Treasure Data REST API.

Description

API call (or REST call)

get a list of jobs

job/list

get a list of jobs with a specified Job ID

job/show/:id

create a job of a given type under the specified database

job/issue/:type/:database

stop a specified job ID if it is running

job/kill/:id

get the query result of a specified job

job/result/:id

get the results of the specified job ID

job/result_export/:target_job_id

show the status information of the given job id at last execution as well as other related information

job/status/:id

check the status of the job through domain key

job/status_by_domain_key/:domain_key


job/list

This API gets you a list of all jobs in a database to which your profile has access.

Verb

URI

GET

/v3/job/list(.:format)

Request Parameters

Parameter Name

Required

Type

Default

Description

Example

from

N

Integer


Gets the Job from the nth index in the list.

By default, the first 20 jobs in the list are displayed.

1

to

N

Integer


Gets the Job up to the nth index in the list.

By default, the first 20 jobs in the list are displayed.

10

slower_than

N

Integer


Condition for TIMESTAMPDIFF() to search for slow queries.

Avoid using this parameter as it can be dangerous.


Sample Request

$ curl -X GET \ https://api.treasuredata.com/v3/job/list?from=1&to=3 \ -H 'Authorization: TD1 <API Key>'

Sample Response Success

{ "jobs": [ { "query": "-- C_BRANCH=\"master\"\n-- td-data-workflow.monitoring.HiveSelect1\nSELECT time FROM www_access LIMIT 1;\n", "type": "hive", "priority": 0, "retry_limit": 0, "duration": 25, "status": "success", "cpu_time": 2330, "result_size": 26, "job_id": "39016393", "created_at": "2019-02-05 08:24:45 UTC", "updated_at": "2019-02-05 08:25:10 UTC", "start_at": "2019-02-05 08:24:45 UTC", "end_at": "2019-02-05 08:25:10 UTC", "num_records": 1, "database": "sample_datasets", "user_name": "Kohta", "result": "", "url": "https://treasuredata.com/jobs/39016393", "hive_result_schema": "[[\"time\", \"int\"]]", "organization": null, "linked_result_export_job_id": null, "result_export_target_job_id": null }, { "query": "-- C_BRANCH=\"master\"\n-- td-data-workflow.monitoring.PrestoSelect1\nSELECT time FROM www_access LIMIT 1;\n", "type": "presto", "priority": 0, "retry_limit": 0, "duration": 1, "status": "success", "cpu_time": null, "result_size": 26, "job_id": "39016385", "created_at": "2019-02-05 08:24:23 UTC", "updated_at": "2019-02-05 08:24:24 UTC", "start_at": "2019-02-05 08:24:23 UTC", "end_at": "2019-02-05 08:24:24 UTC", "num_records": 1, "database": "sample_datasets", "user_name": "Kohta", "result": "", "url": "https://treasuredata.com/jobs/39016385", "hive_result_schema": "[[\"time\", \"bigint\"]]", "organization": null, "linked_result_export_job_id": null, "result_export_target_job_id": null }, { "query": "-- C_BRANCH=\"master\"\n-- td-data-workflow.monitoring.HiveSelect1\nSELECT time FROM www_access LIMIT 1;\n", "type": "hive", "priority": 0, "retry_limit": 0, "duration": 22, "status": "success", "cpu_time": 2630, "result_size": 26, "job_id": "39016308", "created_at": "2019-02-05 08:19:24 UTC", "updated_at": "2019-02-05 08:19:46 UTC", "start_at": "2019-02-05 08:19:24 UTC", "end_at": "2019-02-05 08:19:46 UTC", "num_records": 1, "database": "sample_datasets", "user_name": "Kohta", "result": "", "url": "https://treasuredata.com/jobs/39016308", "hive_result_schema": "[[\"time\", \"int\"]]", "organization": null, "linked_result_export_job_id": null, "result_export_target_job_id": null } ], "count": 3, "from": 1, "to": 3 }

job/show/:id

This API gets the list of jobs with a specified Job ID.

Verb

URI

GET

/v3/job/show/:id(.:format)

URI parameters

URI Parameter

Description

id

Job ID

Sample Request

$ curl -X GET \ https://api.treasuredata.com/v3/job/show/39015859/ \ -H 'Authorization: TD1 <API Key>'

Sample Response Success

{ "query": "-- C_BRANCH=\"master\"\n-- td-data-workflow.monitoring.PrestoSelect1\nSELECT time FROM www_access LIMIT 1;\n", "type": "presto", "priority": 0, "retry_limit": 0, "duration": 1, "status": "success", "cpu_time": null, "result_size": 26, "job_id": "39015859", "created_at": "2019-02-05 07:59:20 UTC", "updated_at": "2019-02-05 07:59:21 UTC", "start_at": "2019-02-05 07:59:20 UTC", "end_at": "2019-02-05 07:59:21 UTC", "num_records": 1, "database": "sample_datasets", "user_name": "Kohta", "result": "", "url": "https://console.treasuredata.com/jobs/39015859", "hive_result_schema": "[[\"time\", \"bigint\"]]", "organization": null, "linked_result_export_job_id": null, "result_export_target_job_id": null, "debug": { "cmdout": "started at 2019-02-05T07:59:20Z\nexecuting query: -- C_BRANCH=\"master\"\n-- td-data-workflow.monitoring.PrestoSelect1\nSELECT time FROM www_access LIMIT 1\nStarted fetching results.\n1 rows.\nQuery plan:\n- Stage-0 (SINGLE / SINGLE)\n -> Output[6]\n Columns: time = time:bigint\n -> Limit[46]\n Count: 1\n -> Exchange[111]\n -> RemoteSource[90]\n Sources: Stage-1\n- Stage-1 (SOURCE)\n -> Limit[89]\n Count: 1\n -> TableScan[0]\n Table: sample_datasets.www_access\n Columns: time:bigint = time:\"time\"\n ** Time indexes:\n Time index: [1970-01-01 00:00:00 UTC, 9999-12-31 23:59:59 UTC]\n2019-02-05 07:59:21 -- memory:0B, peak memory:0B, queued time:179.56ms\n20190205_075921_12078_hm38a 307.18ms rows bytes bytes/sec done total \n[0] FINISHED 1 31B 10.3KB/s 5 / 5 \n [1] FINISHED 2,930 6.3KB 2.1MB/s 10 / 10 \nfinished at 2019-02-05T07:59:21Z\n", "stderr": null } }

job/issue/:type/:database

This API creates a job of a given type under the specified database.

Verb

URI

POST

/v3/job/issue/:type/:database(.:format)

URI parameters

URI Parameter

Description

type - can be the following values

hive

presto

type of query

database

database name

format

details of the job

Sample Request

$ curl -X POST \ https://api.treasuredata.com/v3/job/issue/presto/t_test_db?query=select%201; \ -H 'Authorization: TD1 <API Key>'

Sample Response Success

{ "job": "40962831", "database": "t_test_db", "job_id": "40962831" }

Sample Request with Failure

$ curl -X POST \ https://api.treasuredata.com/v3/job/issue/presto/t_test_db_1?query=select%201; \ -H 'Authorization: TD1 <API Key>'

Sample Response with Failure

{ "error": "Resource not found", "message": "Resource not found", "text": "Resource not found", "severity": "error", "status_code": 404 }

job/kill/:id

This API stops a specified job id if it is running.

Verb

URI

POST

/v3/job/kill/:id(.:format)

URI parameters

URI Parameter

Description

id

Job ID


Sample Request

$ curl -X POST \ https://api.treasuredata.com/v3/job/kill/40494248 \ -H 'Authorization: TD1 <API Key>'

Sample Response Success

{ "error": "Access denied", "message": "Access denied", "text": "Access denied", "severity": "error", "status_code": 403 }

job/result/:id

This API gets the query result of a specified job.

Verb

URI

GET

/v3/job/result/:id(.:format)

URI parameters

URI Parameter

Description

id

Job ID

Sample Request

$ curl -X GET \ https://api.treasuredata.com/v3/job/result/41366080 \ -H 'Authorization: TD1 <API Key>'

Sample Response Success

247eed05c1cce9ad7f88e6ed185755ab,12345678,login,/api/v1/login,iPhone,"Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A403 Safari/8536.25",POST,400390,1448527315e7d0d6dde9658aa7a4d0074bb87f499f,12345678,login,/api/v1/login,iPhone,"Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A403 Safari/8536.25",POST,259377,144852731517c58f03a4a806b8f545a43134e41fa7,12345678,login,/api/v1/login,iPhone,"Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A403 Safari/8536.25",POST,933810,14485273153564b6ec48acb8367bde0a50eceae67e,12345678,login,/api/v1/login,iPhone,"Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A403 Safari/8536.25",POST,737033,14485273151d7115a89559b0e79b16ae662c7cff21,12345678,login,/api/v1/login,iPhone,"Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A403 Safari/8536.25",POST,763460,1448527315661888f02ae52eecf61a71f88b3aade2,12345678,login,/api/v1/login,iPhone,"Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A403 Safari/8536.25",POST,348847,1448527315c3e1ef8bbde538a81b4134d60988c2e3,12345678,login,/api/v1/login,iPhone,"Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A403 Safari/8536.25",POST,911495,1448527315ba19d508278dc3c0517c46de2080bf83,12345678,login,/api/v1/login,iPhone,"Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A403 Safari/8536.25",POST,720568,1448527315a637f1ff156f3ae92dd7a728c768eb05,12345678,login,/api/v1/login,iPhone,"Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A403 Safari/8536.25",POST,669409,1448527315e8109a03890b7b1856c0a96f49ca3085,12345678,login,/api/v1/login,iPhone,"Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A403 Safari/8536.25",POST,156401,14485273154051380ebf42c7cd9019f710605e82f5,12345678,login,/api/v1/login,iPhone,"Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A403 Safari/8536.25",POST,676791,1448527315b131e1c7f49fd8e45df78ef62fcd19c7,12345678,login,/api/v1/login,iPhone,"Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A403 Safari/8536.25",POST,352993,1448527315

Note:

The job results are returned in JSONL (http://jsonlines.org/) format, not standard JSON. The reason for this is that this endpoint needs to return content in a streaming manner, which normal JSON is ill suited for. A streaming format allows the endpoint to cater for large query results. For example, using standard JSON, if a query result had 1 million records, the JSON parser at client side cannot free the memory required until it meets the closing "}", which isn't practical for large data sets. You will need to use a library that handles JSONL format to process the query results.

job/result_export/:target_job_id

This API gets the results of the specified job id.

job/result_export/:target_job_id

Verb

URI

POST

/v3/job/result_export/:target_job_id(.:format)

URI parameters

URI Parameter

Description

target_job_id

job ID for which results will be exported

Sample Request

$ curl -X POST \ https://api.treasuredata.com/v3/job/result_export/40959889 \ -H 'Authorization: TD1 <API Key>

Sample Response Success

{ "query": "RESULT EXPORT FROM JOB 40959889",
"type": "result_export",
"priority": 0,
"retry_limit": 0,
"duration": null, 
"status": "queued", 
"cpu_time": null, 
"result_size": null, 
"job_id": "40960228", 
"created_at": "2019-03-12 02:52:43 UTC", 
"updated_at": "2019-03-12 02:52:43 UTC", 
"start_at": "", 
"end_at": "", 
"num_records": null, 
"database": "stream_import_load_test", 
"user_name": "Kirstin Cardones", 
"result": "", 
"url": "https://console.treasuredata.com/jobs/40960228", 
"hive_result_schema": 
"[[\"_col0\", \"bigint\"]]", 
"organization": null, 
"linked_result_export_job_id": null, 
"result_export_target_job_id": 40959889, 
"debug": { "cmdout": null, "stderr": null } 
}

Sample Request with Failure

$ curl -X POST \ https://api.treasuredata.com/v3/job/result_export/252085 \ -H 'Authorization: TD1 <API Key>

Sample Response with Failure

{ "error": "Job with id \"252085\" not found", "message": "Job with id \"252085\" not found", "text": "Job with id \"252085\" not found", "severity": "error", "status_code": 404 }

job/status/:id

API that shows the status information of the given job id at last execution as well as other related information.

Verb

URI

GET

/v3/job/status/:id(.:format)

URI parameters

URI Parameter

Description

id

Job ID

Sample Request

$ curl -X GET \ https://api.treasuredata.com/v3/job/status/40498813 \ -H 'Authorization: TD1 <API Key>'

Example Response Success

{ "status": "success", "cpu_time": null, "result_size": 24, "duration": 0, "job_id": "40498813", "created_at": "2019-03-04 05:32:22 UTC", "updated_at": "2019-03-04 05:32:23 UTC", "start_at": "2019-03-04 05:32:23 UTC", "end_at": "2019-03-04 05:32:23 UTC", "num_records": 1 }

job/status_by_domain_key/:domain_key

This API checks the status of the job through domain key.

Verb

URI

GET

/v3/job/status_by_domain_key/:domain_key(.:format)

URI parameters

URI Parameter

URI Description

domain_key

job domain key which is assigned to a single job

Sample Request

$ curl -X GET \ https://api.treasuredata.com/v3/job/status_by_domain_key/20190319 \ -H 'Authorization: TD1 <API Key>

Sample Response Success

{ "status": "running", "cpu_time": null, "result_size": null, "duration": null, "job_id": "41365612", "created_at": "2019-03-19 04:39:56 UTC", "updated_at": "2019-03-19 04:39:56 UTC", "start_at": "2019-03-19 04:39:56 UTC", "end_at": "", "num_records": null }

Sample Request with Failure

$ curl -X GET \ https://api.treasuredata.com/v3/job/status_by_domain_key/20190319a \ -H 'Authorization: TD1 <API Key>

Response with Failure

{ "error": "Job with id \"252085\" not found", "message": "Job with id \"252085\" not found", "text": "Job with id \"252085\" not found", "severity": "error", "status_code": 404 }



  • No labels