Visit our new documentation site! This documentation page is no longer updated.

td command line tool reference

This page documents most of td subcommands.

Table of Contents

td db:list

usage:

$ td db:list

example:

$ td db:list
$ td dbs

description:

Show list of tables in a database

options:

-f, --format FORMAT              format of the result rendering (tsv, csv, json or table. default is table)

td db:show

usage:

$ td db:show <db>

example:

$ td db example_db

description:

Describe information of a database

options:

-f, --format FORMAT              format of the result rendering (tsv, csv, json or table. default is table)

td db:create

usage:

$ td db:create <db>

example:

$ td db:create example_db

description:

Create a database

td db:delete

usage:

$ td db:delete <db>

example:

$ td db:delete example_db

description:

Delete a database

options:

-f, --force                      clear tables and delete the database

td export:result

usage:

$ td export:result <id,> <name>

example:

$ td export:result target_job_id result

description:

Dump logs of a target query to the specified place

options:

-w, --wait                       wait until the job is completed
-P, --priority PRIORITY          set priority
-R, --retry COUNT                automatic retrying count

td export:table

usage:

$ td export:table <db> <table>

example:

$ td export:table example_db table1 --s3-bucket mybucket -k KEY_ID -s SECRET_KEY

description:

Dump logs in a table to the specified storage

options:

-w, --wait                       wait until the job is completed
-f, --from TIME                  export data which is newer than or same with the TIME
-t, --to TIME                    export data which is older than the TIME
-b, --s3-bucket NAME             name of the destination S3 bucket (required)
-p, --prefix PATH                path prefix of the file on S3
-k, --aws-key-id KEY_ID          AWS access key id to export data (required)
-s, --aws-secret-key SECRET_KEY  AWS secret access key to export data (required)
-F, --file-format FILE_FORMAT    file format for exported data.
                                 Available formats are tsv.gz (tab-separated values per line) and jsonl.gz (JSON record per line).
                                 The json.gz and line-json.gz formats are default and still available but only for backward compatibility purpose;
                                   use is discouraged because they have far lower performance.
-O, --pool-name NAME             specify resource pool by name
-e, --encryption ENCRYPT_METHOD  export with server side encryption with the ENCRYPT_METHOD
-a ASSUME_ROLE_ARN,              export with assume role with ASSUME_ROLE_ARN as role arn
    --assume-role

td table:list

usage:

$ td table:list [db]

example:

$ td table:list
$ td table:list example_db
$ td tables

description:

Show list of tables

options:

-n, --num_threads VAL            number of threads to get list in parallel
    --show-bytes                 show estimated table size in bytes
-f, --format FORMAT              format of the result rendering (tsv, csv, json or table. default is table)

td table:show

usage:

$ td table:show <db> <table>

example:

$ td table example_db table1

description:

Describe information of a table

options:

-v                               show more attributes

td table:create

usage:

$ td table:create <db> <table>

example:

$ td table:create example_db table1

description:

Create a table

options:

-T, --type TYPE                  set table type (log)
    --expire-days DAYS           set table expire days
    --include-v BOOLEAN          set include_v flag
    --detect-schema BOOLEAN      set detect schema flag

td table:delete

usage:

$ td table:delete <db> <table>

example:

$ td table:delete example_db table1

description:

Delete a table

options:

-f, --force                      never prompt

td table:import

usage:

$ td table:import <db> <table> <files...>

example:

$ td table:import example_db table1 --apache access.log
$ td table:import example_db table1 --json -t time - < test.json

description:

Parse and import files to a table

options:

    --format FORMAT              file format (default: apache)
    --apache                     same as --format apache; apache common log format
    --syslog                     same as --format syslog; syslog
    --msgpack                    same as --format msgpack; msgpack stream format
    --json                       same as --format json; LF-separated json format
-t, --time-key COL_NAME          time key name for json and msgpack format (e.g. 'created_at')
    --auto-create-table          Create table and database if doesn't exist

td table:export

usage:

$ td table:export <db> <table>

example:

$ td table:export example_db table1 --s3-bucket mybucket -k KEY_ID -s SECRET_KEY

description:

Dump logs in a table to the specified storage

options:

-w, --wait                       wait until the job is completed
-f, --from TIME                  export data which is newer than or same with the TIME
-t, --to TIME                    export data which is older than the TIME
-b, --s3-bucket NAME             name of the destination S3 bucket (required)
-p, --prefix PATH                path prefix of the file on S3
-k, --aws-key-id KEY_ID          AWS access key id to export data (required)
-s, --aws-secret-key SECRET_KEY  AWS secret access key to export data (required)
-F, --file-format FILE_FORMAT    file format for exported data.
                                 Available formats are tsv.gz (tab-separated values per line) and jsonl.gz (JSON record per line).
                                 The json.gz and line-json.gz formats are default and still available but only for backward compatibility purpose;
                                   use is discouraged because they have far lower performance.
-O, --pool-name NAME             specify resource pool by name
-e, --encryption ENCRYPT_METHOD  export with server side encryption with the ENCRYPT_METHOD
-a ASSUME_ROLE_ARN,              export with assume role with ASSUME_ROLE_ARN as role arn
    --assume-role

td table:swap

usage:

$ td table:swap <db> <table1> <table2>

example:

$ td table:swap example_db table1 table2

description:

Swap names of two tables

td table:rename

usage:

$ td table:rename <db> <from_table> <dest_table>

example:

$ td table:rename example_db table1 table2

description:

rename exist table

options:

    --overwrite                  replace existing dest table

td table:tail

usage:

$ td table:tail <db> <table>

example:

$ td table:tail example_db table1
$ td table:tail example_db table1 -n 30

description:

Get recently imported logs

options:

-n, --count N                    number of logs to get
-P, --pretty                     pretty print

td table:partial_delete

usage:

$ td table:partial_delete <db> <table>

example:

$ td table:partial_delete example_db table1 --from 1341000000 --to 1341003600

description:

Delete logs from the table within the specified time range

options:

-t, --to TIME                    end time of logs to delete in Unix time >0 and multiple of 3600 (1 hour)
-f, --from TIME                  start time of logs to delete in Unix time >0 and multiple of 3600 (1 hour)
-w, --wait                       wait for the job to finish
-O, --pool-name NAME             specify resource pool by name

td table:expire

usage:

$ td table:expire <db> <table> <expire_days>

example:

$ td table:expire example_db table1 30

description:

Expire data in table after specified number of days. Set to 0 to disable the expiration.

td table:update

usage:

$ td table:update <db> <table>

example:

$ td table:update example_db table1 --include-v false

description:

Update table options

options:

    --expire-days DAYS           set table expire days
    --include-v BOOLEAN          set include_v flag
    --detect-schema BOOLEAN      set detect schema flag

td import:list

usage:

$ td import:list

example:

$ td import:list

description:

List bulk import sessions

options:

-f, --format FORMAT              format of the result rendering (tsv, csv, json or table. default is table)

td import:show

usage:

$ td import:show <name>

example:

$ td import:show

description:

Show list of uploaded parts

td import:create

usage:

$ td import:create <name> <db> <table>

example:

$ td import:create logs_201201 example_db event_logs

description:

Create a new bulk import session to the table

td import:jar_version

usage:

$ td import:jar_version

example:

$ td import:jar_version

description:

Show import jar version

td import:jar_update

usage:

$ td import:jar_update

example:

$ td import:jar_update

description:

Update import jar to the latest version

td import:prepare

usage:

$ td import:prepare <files...>

example:

$ td import:prepare logs/*.csv --format csv --columns time,uid,price,count --time-column time -o parts/
$ td import:prepare logs/*.csv --format csv --columns date_code,uid,price,count --time-value 1394409600,10 -o parts/
$ td import:prepare mytable --format mysql --db-url jdbc:mysql://localhost/mydb --db-user myuser --db-password mypass
$ td import:prepare "s3://<s3_access_key>:<s3_secret_key>@/my_bucket/path/to/*.csv" --format csv --column-header --time-column date_time -o parts/

description:

Convert files into part file format

options:

  -f, --format FORMAT              source file format [csv, tsv, json, msgpack, apache, regex, mysql]; default=csv
  -C, --compress TYPE              compressed type [gzip, none, auto]; default=auto detect
  -T, --time-format FORMAT         specifies the strftime format of the time column
                                    The format slightly differs from Ruby's Time#strftime format in that the
                                    '%:z' and '%::z' timezone options are not supported.
  -e, --encoding TYPE              encoding type [UTF-8, etc.]
  -o, --output DIR                 output directory. default directory is 'out'.
  -s, --split-size SIZE_IN_KB      size of each parts (default: 16384)
  -t, --time-column NAME           name of the time column
  --time-value TIME,HOURS          time column's value. If the data doesn't have a time column,
                                   users can auto-generate the time column's value in 2 ways:
                                    * Fixed time value with --time-value TIME:
                                      where TIME is a Unix time in seconds since Epoch. The time
                                      column value is constant and equal to TIME seconds.
                                      E.g. '--time-value 1394409600' assigns the equivalent of
                                      timestamp 2014-03-10T00:00:00 to all records imported.
                                    * Incremental time value with --time-value TIME,HOURS:
                                      where TIME is the Unix time in seconds since Epoch and
                                      HOURS is the maximum range of the timestamps in hours.
                                      This mode can be used to assign incremental timestamps to
                                      subsequent records. Timestamps will be incremented by 1 second
                                      each record. If the number of records causes the timestamp to
                                      overflow the range (timestamp >= TIME + HOURS * 3600), the
                                      next timestamp will restart at TIME and continue from there.
                                      E.g. '--time-value 1394409600,10' will assign timestamp 1394409600
                                      to the first record, timestamp 1394409601 to the second, 1394409602
                                      to the third, and so on until the 36000th record which will have
                                      timestamp 1394445600 (1394409600 + 10 * 3600). The timestamp assigned
                                      to the 36001th record will be 1394409600 again and the timestamp
                                      will restart from there.
  --primary-key NAME:TYPE          pair of name and type of primary key declared in your item table
  --prepare-parallel NUM           prepare in parallel (default: 2; max 96)
  --only-columns NAME,NAME,...     only columns
  --exclude-columns NAME,NAME,...  exclude columns
  --error-records-handling MODE    error records handling mode [skip, abort]; default=skip
  --invalid-columns-handling MODE  invalid columns handling mode [autofix, warn]; default=warn
  --error-records-output DIR       write error records; default directory is 'error-records'.
  --columns NAME,NAME,...          column names (use --column-header instead if the first line has column names)
  --column-types TYPE,TYPE,...     column types [string, int, long, double]
  --column-type NAME:TYPE          column type [string, int, long, double]. A pair of column name and type can be specified like 'age:int'
  -S, --all-string                 disable automatic type conversion
  --empty-as-null-if-numeric       the empty string values are interpreted as null values if columns are numerical types.

  CSV/TSV specific options:
  --column-header                  first line includes column names
  --delimiter CHAR                 delimiter CHAR; default="," at csv, "\t" at tsv
  --escape CHAR                    escape CHAR; default=\
  --newline TYPE                   newline [CRLF, LF, CR];  default=CRLF
  --quote CHAR                     quote [DOUBLE, SINGLE, NONE]; if csv format, default=DOUBLE. if tsv format, default=NONE

  MySQL specific options:
  --db-url URL                     JDBC connection URL
  --db-user NAME                   user name for MySQL account
  --db-password PASSWORD           password for MySQL account

  REGEX specific options:
  --regex-pattern PATTERN          pattern to parse line. When 'regex' is used as source file format, this option is required

td import:upload

usage:

$ td import:upload <session name> <files...>

example:

$ td import:upload mysess parts/* --parallel 4
$ td import:upload mysess parts/*.csv --format csv --columns time,uid,price,count --time-column time -o parts/
$ td import:upload parts/*.csv --auto-create mydb.mytbl --format csv --columns time,uid,price,count --time-column time -o parts/
$ td import:upload mysess mytable --format mysql --db-url jdbc:mysql://localhost/mydb --db-user myuser --db-password mypass
$ td import:upload "s3://<s3_access_key>:<s3_secret_key>@/my_bucket/path/to/*.csv" --format csv --column-header --time-column date_time -o parts/

description:

Upload or re-upload files into a bulk import session

options:

  --retry-count NUM                upload process will automatically retry at specified time; default: 10
  --auto-create DATABASE.TABLE     create automatically bulk import session by specified database and table names
                                   If you use 'auto-create' option, you MUST not specify any session name as first argument.
  --auto-perform                   perform bulk import job automatically
  --auto-commit                    commit bulk import job automatically
  --auto-delete                    delete bulk import session automatically
  --parallel NUM                   upload in parallel (default: 2; max 8)

  -f, --format FORMAT              source file format [csv, tsv, json, msgpack, apache, regex, mysql]; default=csv
  -C, --compress TYPE              compressed type [gzip, none, auto]; default=auto detect
  -T, --time-format FORMAT         specifies the strftime format of the time column
                                    The format slightly differs from Ruby's Time#strftime format in that the
                                    '%:z' and '%::z' timezone options are not supported.
  -e, --encoding TYPE              encoding type [UTF-8, etc.]
  -o, --output DIR                 output directory. default directory is 'out'.
  -s, --split-size SIZE_IN_KB      size of each parts (default: 16384)
  -t, --time-column NAME           name of the time column
  --time-value TIME,HOURS          time column's value. If the data doesn't have a time column,
                                   users can auto-generate the time column's value in 2 ways:
                                    * Fixed time value with --time-value TIME:
                                      where TIME is a Unix time in seconds since Epoch. The time
                                      column value is constant and equal to TIME seconds.
                                      E.g. '--time-value 1394409600' assigns the equivalent of
                                      timestamp 2014-03-10T00:00:00 to all records imported.
                                    * Incremental time value with --time-value TIME,HOURS:
                                      where TIME is the Unix time in seconds since Epoch and
                                      HOURS is the maximum range of the timestamps in hours.
                                      This mode can be used to assign incremental timestamps to
                                      subsequent records. Timestamps will be incremented by 1 second
                                      each record. If the number of records causes the timestamp to
                                      overflow the range (timestamp >= TIME + HOURS * 3600), the
                                      next timestamp will restart at TIME and continue from there.
                                      E.g. '--time-value 1394409600,10' will assign timestamp 1394409600
                                      to the first record, timestamp 1394409601 to the second, 1394409602
                                      to the third, and so on until the 36000th record which will have
                                      timestamp 1394445600 (1394409600 + 10 * 3600). The timestamp assigned
                                      to the 36001th record will be 1394409600 again and the timestamp
                                      will restart from there.
  --primary-key NAME:TYPE          pair of name and type of primary key declared in your item table
  --prepare-parallel NUM           prepare in parallel (default: 2; max 96)
  --only-columns NAME,NAME,...     only columns
  --exclude-columns NAME,NAME,...  exclude columns
  --error-records-handling MODE    error records handling mode [skip, abort]; default=skip
  --invalid-columns-handling MODE  invalid columns handling mode [autofix, warn]; default=warn
  --error-records-output DIR       write error records; default directory is 'error-records'.
  --columns NAME,NAME,...          column names (use --column-header instead if the first line has column names)
  --column-types TYPE,TYPE,...     column types [string, int, long, double]
  --column-type NAME:TYPE          column type [string, int, long, double]. A pair of column name and type can be specified like 'age:int'
  -S, --all-string                 disable automatic type conversion
  --empty-as-null-if-numeric       the empty string values are interpreted as null values if columns are numerical types.

  CSV/TSV specific options:
  --column-header                  first line includes column names
  --delimiter CHAR                 delimiter CHAR; default="," at csv, "\t" at tsv
  --escape CHAR                    escape CHAR; default=\
  --newline TYPE                   newline [CRLF, LF, CR];  default=CRLF
  --quote CHAR                     quote [DOUBLE, SINGLE, NONE]; if csv format, default=DOUBLE. if tsv format, default=NONE

  MySQL specific options:
  --db-url URL                     JDBC connection URL
  --db-user NAME                   user name for MySQL account
  --db-password PASSWORD           password for MySQL account

  REGEX specific options:
  --regex-pattern PATTERN          pattern to parse line. When 'regex' is used as source file format, this option is required

td import:auto

usage:

$ td import:auto <session name> <files...>

example:

$ td import:auto mysess parts/* --parallel 4
$ td import:auto mysess parts/*.csv --format csv --columns time,uid,price,count --time-column time -o parts/
$ td import:auto parts/*.csv --auto-create mydb.mytbl --format csv --columns time,uid,price,count --time-column time -o parts/
$ td import:auto mysess mytable --format mysql --db-url jdbc:mysql://localhost/mydb --db-user myuser --db-password mypass
$ td import:auto "s3://<s3_access_key>:<s3_secret_key>@/my_bucket/path/to/*.csv" --format csv --column-header --time-column date_time -o parts/

description:

Automatically upload or re-upload files into a bulk import session. It's functional equivalent of 'upload' command with 'auto-perform', 'auto-commit' and 'auto-delete' options. But it, by default, doesn't provide 'auto-create' option. If you want 'auto-create' option, you explicitly must declare it as command options.

options:

  --retry-count NUM                upload process will automatically retry at specified time; default: 10
  --auto-create DATABASE.TABLE     create automatically bulk import session by specified database and table names
                                   If you use 'auto-create' option, you MUST not specify any session name as first argument.
  --parallel NUM                   upload in parallel (default: 2; max 8)

  -f, --format FORMAT              source file format [csv, tsv, json, msgpack, apache, regex, mysql]; default=csv
  -C, --compress TYPE              compressed type [gzip, none, auto]; default=auto detect
  -T, --time-format FORMAT         specifies the strftime format of the time column
                                    The format slightly differs from Ruby's Time#strftime format in that the
                                    '%:z' and '%::z' timezone options are not supported.
  -e, --encoding TYPE              encoding type [UTF-8, etc.]
  -o, --output DIR                 output directory. default directory is 'out'.
  -s, --split-size SIZE_IN_KB      size of each parts (default: 16384)
  -t, --time-column NAME           name of the time column
  --time-value TIME,HOURS          time column's value. If the data doesn't have a time column,
                                   users can auto-generate the time column's value in 2 ways:
                                    * Fixed time value with --time-value TIME:
                                      where TIME is a Unix time in seconds since Epoch. The time
                                      column value is constant and equal to TIME seconds.
                                      E.g. '--time-value 1394409600' assigns the equivalent of
                                      timestamp 2014-03-10T00:00:00 to all records imported.
                                    * Incremental time value with --time-value TIME,HOURS:
                                      where TIME is the Unix time in seconds since Epoch and
                                      HOURS is the maximum range of the timestamps in hours.
                                      This mode can be used to assign incremental timestamps to
                                      subsequent records. Timestamps will be incremented by 1 second
                                      each record. If the number of records causes the timestamp to
                                      overflow the range (timestamp >= TIME + HOURS * 3600), the
                                      next timestamp will restart at TIME and continue from there.
                                      E.g. '--time-value 1394409600,10' will assign timestamp 1394409600
                                      to the first record, timestamp 1394409601 to the second, 1394409602
                                      to the third, and so on until the 36000th record which will have
                                      timestamp 1394445600 (1394409600 + 10 * 3600). The timestamp assigned
                                      to the 36001th record will be 1394409600 again and the timestamp
                                      will restart from there.
  --primary-key NAME:TYPE          pair of name and type of primary key declared in your item table
  --prepare-parallel NUM           prepare in parallel (default: 2; max 96)
  --only-columns NAME,NAME,...     only columns
  --exclude-columns NAME,NAME,...  exclude columns
  --error-records-handling MODE    error records handling mode [skip, abort]; default=skip
  --invalid-columns-handling MODE  invalid columns handling mode [autofix, warn]; default=warn
  --error-records-output DIR       write error records; default directory is 'error-records'.
  --columns NAME,NAME,...          column names (use --column-header instead if the first line has column names)
  --column-types TYPE,TYPE,...     column types [string, int, long, double]
  --column-type NAME:TYPE          column type [string, int, long, double]. A pair of column name and type can be specified like 'age:int'
  -S, --all-string                 disable automatic type conversion
  --empty-as-null-if-numeric       the empty string values are interpreted as null values if columns are numerical types.

  CSV/TSV specific options:
  --column-header                  first line includes column names
  --delimiter CHAR                 delimiter CHAR; default="," at csv, "\t" at tsv
  --escape CHAR                    escape CHAR; default=\
  --newline TYPE                   newline [CRLF, LF, CR];  default=CRLF
  --quote CHAR                     quote [DOUBLE, SINGLE, NONE]; if csv format, default=DOUBLE. if tsv format, default=NONE

  MySQL specific options:
  --db-url URL                     JDBC connection URL
  --db-user NAME                   user name for MySQL account
  --db-password PASSWORD           password for MySQL account

  REGEX specific options:
  --regex-pattern PATTERN          pattern to parse line. When 'regex' is used as source file format, this option is required

td import:perform

usage:

$ td import:perform <name>

example:

$ td import:perform logs_201201

description:

Start to validate and convert uploaded files

options:

-w, --wait                       wait for finishing the job
-f, --force                      force start performing
-O, --pool-name NAME             specify resource pool by name

td import:error_records

usage:

$ td import:error_records <name>

example:

$ td import:error_records logs_201201

description:

Show records which did not pass validations

td import:commit

usage:

$ td import:commit <name>

example:

$ td import:commit logs_201201

description:

Start to commit a performed bulk import session

options:

-w, --wait                       wait for finishing the commit

td import:delete

usage:

$ td import:delete <name>

example:

$ td import:delete logs_201201

description:

Delete a bulk import session

td import:freeze

usage:

$ td import:freeze <name>

example:

$ td import:freeze logs_201201

description:

Reject succeeding uploadings to a bulk import session

td import:unfreeze

usage:

$ td import:unfreeze <name>

example:

$ td import:unfreeze logs_201201

description:

Unfreeze a frozen bulk import session

td import:config

usage:

$ td import:config <files...>

example:

$ td import:config "s3://<s3_access_key>:<s3_secret_key>@/my_bucket/path/to/*.csv" -o seed.yml

description:

create guess config from arguments

options:

-o, --out FILE_NAME              output file name for connector:guess
-f, --format FORMAT              source file format [csv, tsv, mysql]; default=csv
    --db-url URL                 Database Connection URL
    --db-user NAME               user name for database
    --db-password PASSWORD       password for database
    --columns COLUMNS            not supported
    --column-header COLUMN-HEADER
                                 not supported
    --time-column TIME-COLUMN    not supported
    --time-format TIME-FORMAT    not supported

td result:list

usage:

$ td result:list

example:

$ td result:list
$ td results

description:

Show list of result URLs

options:

-f, --format FORMAT              format of the result rendering (tsv, csv, json or table. default is table)

td result:show

usage:

$ td result:show <name>

example:

$ td result name

description:

Describe information of a result URL

td result:create

usage:

$ td result:create <name> <URL>

example:

$ td result:create name mysql://my-server/mydb

description:

Create a result URL

options:

-u, --user NAME                  set user name for authentication
-p, --password                   ask password for authentication

td result:delete

usage:

$ td result:delete <name>

example:

$ td result:delete name

description:

Delete a result URL

td schema:show

usage:

$ td schema:show <db> <table>

example:

$ td schema example_db table1

description:

Show schema of a table

td schema:set

usage:

$ td schema:set <db> <table> [columns...]

example:

$ td schema:set example_db table1 user:string size:int

description:

Set new schema on a table

td schema:add

usage:

$ td schema:add <db> <table> <columns...>

example:

$ td schema:add example_db table1 user:string size:int

description:

Add new columns to a table

td schema:remove

usage:

$ td schema:remove <db> <table> <columns...>

example:

$ td schema:remove example_db table1 user size

description:

Remove columns from a table

td sched:list

usage:

$ td sched:list

example:

$ td sched:list
$ td scheds

description:

Show list of schedules

options:

-f, --format FORMAT              format of the result rendering (tsv, csv, json or table. default is table)

td sched:create

usage:

$ td sched:create <name> <cron> [sql]

example:

$ td sched:create sched1 "0 * * * *" -d example_db "select count(*) from table1" -r rset1
$ td sched:create sched1 "0 * * * *" -d example_db -q query.txt -r rset2

description:

Create a schedule

options:

-d, --database DB_NAME           use the database (required)
-t, --timezone TZ                name of the timezone.
                                   Only extended timezones like 'Asia/Tokyo', 'America/Los_Angeles' are supported,
                                   (no 'PST', 'PDT', etc...).
                                   When a timezone is specified, the cron schedule is referred to that timezone.
                                   Otherwise, the cron schedule is referred to the UTC timezone.
                                   E.g. cron schedule '0 12 * * *' will execute daily at 5 AM without timezone option
                                   and at 12PM with the -t / --timezone 'America/Los_Angeles' timezone option
-D, --delay SECONDS              delay time of the schedule
-r, --result RESULT_URL          write result to the URL (see also result:create subcommand)
-u, --user NAME                  set user name for the result URL
-p, --password                   ask password for the result URL
-P, --priority PRIORITY          set priority
-q, --query PATH                 use file instead of inline query
-R, --retry COUNT                automatic retrying count
-T, --type TYPE                  set query type (hive)

td sched:delete

usage:

$ td sched:delete <name>

example:

$ td sched:delete sched1

description:

Delete a schedule

td sched:update

usage:

$ td sched:update <name>

example:

$ td sched:update sched1 -s "0 */2 * * *" -d my_db -t "Asia/Tokyo" -D 3600

description:

Modify a schedule

options:

-n, --newname NAME               change the schedule's name
-s, --schedule CRON              change the schedule
-q, --query SQL                  change the query
-d, --database DB_NAME           change the database
-r, --result RESULT_URL          change the result target (see also result:create subcommand)
-t, --timezone TZ                name of the timezone.
                                   Only extended timezones like 'Asia/Tokyo', 'America/Los_Angeles' are supported,
                                   (no 'PST', 'PDT', etc...).
                                   When a timezone is specified, the cron schedule is referred to that timezone.
                                   Otherwise, the cron schedule is referred to the UTC timezone.
                                   E.g. cron schedule '0 12 * * *' will execute daily at 5 AM without timezone option
                                   and at 12PM with the -t / --timezone 'America/Los_Angeles' timezone option
-D, --delay SECONDS              change the delay time of the schedule
-P, --priority PRIORITY          set priority
-R, --retry COUNT                automatic retrying count
-T, --type TYPE                  set query type (hive)

td sched:history

usage:

$ td sched:history <name> [max]

example:

$ td sched sched1 --page 1

description:

Show history of scheduled queries

options:

-p, --page PAGE                  skip N pages
-s, --skip N                     skip N schedules
-f, --format FORMAT              format of the result rendering (tsv, csv, json or table. default is table)

td sched:run

usage:

$ td sched:run <name> <time>

example:

$ td sched:run sched1 "2013-01-01 00:00:00" -n 6

description:

Run scheduled queries for the specified time

options:

-n, --num N                      number of jobs to run
-f, --format FORMAT              format of the result rendering (tsv, csv, json or table. default is table)

td sched:result

usage:

$ td sched:result <name>

example:

$ td sched:result NAME | sched:result NAME --last | sched:result NAME --last 3

description:

Show status and result of the last job ran.
--last [N] option enables to show the result before N from the last.
The other options are identical to those of the 'job:show' command.

options:

-v, --verbose                    show logs
-w, --wait                       wait for finishing the job
-G, --vertical                   use vertical table to show results
-o, --output PATH                write result to the file
-l, --limit ROWS                 limit the number of result rows shown when not outputting to file
-c, --column-header              output of the columns' header when the schema is available
                                   for the table (only applies to tsv and csv formats)
-x, --exclude                    do not automatically retrieve the job result
    --null STRING                null expression in csv or tsv
-f, --format FORMAT              format of the result to write to the file (tsv, csv, json, msgpack, and msgpack.gz)
    --last [Number]              show the result before N from the last. default: 1

td query

usage:

$ td query [sql]

example:

$ td query -d example_db -w -r rset1 "select count(*) from table1"
$ td query -d example_db -w -r rset1 -q query.txt

description:

Issue a query

options:

-d, --database DB_NAME           use the database (required)
-w, --wait[=SECONDS]             wait for finishing the job (for seconds)
-G, --vertical                   use vertical table to show results
-o, --output PATH                write result to the file
-f, --format FORMAT              format of the result to write to the file (tsv, csv, json, msgpack, and msgpack.gz)
-r, --result RESULT_URL          write result to the URL (see also result:create subcommand)
                                  It is suggested for this option to be used with the -x / --exclude option to suppress printing
                                  of the query result to stdout or -o / --output to dump the query result into a file.
-u, --user NAME                  set user name for the result URL
-p, --password                   ask password for the result URL
-P, --priority PRIORITY          set priority
-R, --retry COUNT                automatic retrying count
-q, --query PATH                 use file instead of inline query
-T, --type TYPE                  set query type (hive, presto)
    --sampling DENOMINATOR       OBSOLETE - enable random sampling to reduce records 1/DENOMINATOR
-l, --limit ROWS                 limit the number of result rows shown when not outputting to file
-c, --column-header              output of the columns' header when the schema is available for the table (only applies to json, tsv and csv formats)
-x, --exclude                    do not automatically retrieve the job result
-O, --pool-name NAME             specify resource pool by name
    --domain-key DOMAIN_KEY      optional user-provided unique ID. You can include this ID with your `create` request to ensure idempotence

td job:show

usage:

$ td job:show <job_id>

example:

$ td job:show 1461

description:

Show status and result of a job

options:

-v, --verbose                    show logs
-w, --wait                       wait for finishing the job
-G, --vertical                   use vertical table to show results
-o, --output PATH                write result to the file
-l, --limit ROWS                 limit the number of result rows shown when not outputting to file
-c, --column-header              output of the columns' header when the schema is available
                                   for the table (only applies to tsv and csv formats)
-x, --exclude                    do not automatically retrieve the job result
    --null STRING                null expression in csv or tsv
-f, --format FORMAT              format of the result to write to the file (tsv, csv, json, msgpack, and msgpack.gz)

td job:status

usage:

$ td job:status <job_id>

example:

$ td job:status 1461

description:

Show status progress of a job

td job:list

usage:

$ td job:list [max]

example:

$ td jobs
$ td jobs --page 1

description:

Show list of jobs

options:

-p, --page PAGE                  skip N pages
-s, --skip N                     skip N jobs
-R, --running                    show only running jobs
-S, --success                    show only succeeded jobs
-E, --error                      show only failed jobs
    --slow [SECONDS]             show slow queries (default threshold: 3600 seconds)
-f, --format FORMAT              format of the result rendering (tsv, csv, json or table. default is table)

td job:kill

usage:

$ td job:kill <job_id>

example:

$ td job:kill 1461

description:

Kill or cancel a job

td password:change

usage:

$ td password:change

description:

Change password

td server:status

usage:

$ td server:status

description:

Show status of the Treasure Data server

td server:endpoint

usage:

$ td server:endpoint <api_endpoint>

example:

$ td server:endpoint 'https://api.treasuredata.com'

description:

Set the Treasure Data API server's endpoint (must be a valid URI)

td sample:apache

usage:

$ td sample:apache <path.json>

description:

Create a sample log file

td connector:guess

usage:

$ td connector:guess [config]

example:

$ td connector:guess config.yml -o td-bulkload.yml

example config.yml:

in:
  type: s3
  bucket: my-s3-bucket
  endpoint: s3-us-west-1.amazonaws.com
  path_prefix: path/prefix/to/import/
  access_key_id: ABCXYZ123ABCXYZ123
  secret_access_key: AbCxYz123aBcXyZ123
out:
  mode: append

description:

Run guess to generate connector config file

options:

    --type[=TYPE]                (obsoleted)
    --access-id ID               (obsoleted)
    --access-secret SECRET       (obsoleted)
    --source SOURCE              (obsoleted)
-o, --out FILE_NAME              output file name for connector:preview
-g, --guess NAME,NAME,...        specify list of guess plugins that users want to use

td connector:preview

usage:

$ td connector:preview <config>

example:

$ td connector:preview td-bulkload.yml

description:

Show preview of connector execution

options:

-f, --format FORMAT              format of the result rendering (tsv, csv, json or table. default is table)

td connector:issue

usage:

$ td connector:issue <config>

example:

$ td connector:issue td-bulkload.yml

description:

Run one time connector execution

options:

    --database DB_NAME           destination database
    --table TABLE_NAME           destination table
    --time-column COLUMN_NAME    data partitioning key
-w, --wait                       wait for finishing the job
    --auto-create-table          Create table and database if doesn't exist

td connector:list

usage:

$ td connector:list

example:

$ td connector:list

description:

Show list of connector sessions

options:

-f, --format FORMAT              format of the result rendering (tsv, csv, json or table. default is table)

td connector:create

usage:

$ td connector:create <name> <cron> <database> <table> <config>

example:

$ td connector:create connector1 "0 * * * *" connector_database connector_table td-bulkload.yml

description:

Create new connector session

options:

    --time-column COLUMN_NAME    data partitioning key
-t, --timezone TZ                name of the timezone.
                                   Only extended timezones like 'Asia/Tokyo', 'America/Los_Angeles' are supported,
                                   (no 'PST', 'PDT', etc...).
                                   When a timezone is specified, the cron schedule is referred to that timezone.
                                   Otherwise, the cron schedule is referred to the UTC timezone.
                                   E.g. cron schedule '0 12 * * *' will execute daily at 5 AM without timezone option
                                   and at 12PM with the -t / --timezone 'America/Los_Angeles' timezone option
-D, --delay SECONDS              delay time of the schedule

td connector:show

usage:

$ td connector:show <name>

example:

$ td connector:show connector1

description:

Show connector session

td connector:update

usage:

$ td connector:update <name> [config]

example:

$ td connector:update connector1 -c td-bulkload.yml -s '@daily' ...

description:

Modify connector session

options:

-n, --newname NAME               change the schedule's name
-d, --database DB_NAME           change the database
-t, --table TABLE_NAME           change the table
-s, --schedule [CRON]            change the schedule or leave blank to remove the schedule
-z, --timezone TZ                name of the timezone.
                                   Only extended timezones like 'Asia/Tokyo', 'America/Los_Angeles' are supported,
                                   (no 'PST', 'PDT', etc...).
                                   When a timezone is specified, the cron schedule is referred to that timezone.
                                   Otherwise, the cron schedule is referred to the UTC timezone.
                                   E.g. cron schedule '0 12 * * *' will execute daily at 5 AM without timezone option
                                   and at 12PM with the -t / --timezone 'America/Los_Angeles' timezone option
-D, --delay SECONDS              change the delay time of the schedule
-T, --time-column COLUMN_NAME    change the name of the time column
-c, --config CONFIG_FILE         update the connector configuration
    --config-diff CONFIG_DIFF_FILE
                                 update the connector config_diff

td connector:delete

usage:

$ td connector:delete <name>

example:

$ td connector:delete connector1

description:

Delete connector session

td connector:history

usage:

$ td connector:history <name>

example:

$ td connector:history connector1

description:

Show job history of connector session

options:

-f, --format FORMAT              format of the result rendering (tsv, csv, json or table. default is table)

td connector:run

usage:

$ td connector:run <name> [time]

example:

$ td connector:run connector1 "2016-01-01 00:00:00"

description:

Run connector with session for the specified time option

options:

-w, --wait                       wait for finishing the job

td workflow

td workflow:reset

usage:

$ td workflow:reset

description:

Reset the workflow module

td workflow:update

usage:

$ td workflow:update [version]

description:

Update the workflow module

td workflow:version

usage:

$ td workflow:version

description:

Show workflow module version

Last modified: Feb 27 2018 21:36:47 UTC

If this article is incorrect or outdated, or omits critical information, let us know. For all other issues, access our support channels.