Skip to content
/
Legacy Bulk Import From M...
Last updated

Legacy Bulk Import From Mysql

This article explains how to import data from MySQL to Treasure Data.

Install Legacy Bulk Import

First, install the Toolbelt, which includes a bulk loader program, on your computer.

Downloads

After the installation, the td command is installed on your computer.

Open the terminal and type td to execute the command.

Make sure you have Java as well.

Execute td import:jar_update to download the up-to-date version of our bulk loader:

$ td
usage: td [options] COMMAND [args]
$ java
Usage: java [-options] class [args...]
$ td import:jar_update
Installed td-import.jar 0.x.xx into /path/to/.td/java

Importing data from MySQL

Run the following commands to upload data from MySQL.

$ td db:create my_db
$ td table:create my_db my_tbl
$ td import:auto \
  --auto-create my_db.my_tbl \
  --format mysql \
  --db-url jdbc:mysql://my_sql_host/my_sql_db \
  --db-user my_user \
  --db-password my_pass \
  --time-column date_time \
  my_sql_tbl
Because td import:auto executes MapReduce jobs to check the invalid rows, it'll take at least 1-2 minutes.

In this example, we assumed that:

  • the bulk import reads the input data from the my_sql_tbl on the MySQL database specified by the ‘--db-’ prefixed options
  • the bulk import session is created automatically by our request using the --auto-create option.
    When using the --auto-create option you must provide the database and table name in the format indicated. The session name will be ‘my_db.my_tbl’.
    Alternatively, you can create the bulk import session with a name of choice, using the td import:create command (for details see Bulk Import Internals, Import Step-by-Step).
  • the time field is called “date_time” and is provided with the --time-column option

You can use the following options to specify the access to the MySQL database:

MySQL specific options:
--db-url URL                     JDBC connection URL
--db-user NAME                   user name for MySQL account
--db-password PASSWORD           password for MySQL account

For further details, check the following pages:

Previous page
Next page