Skip to content
Last updated

Teradata Export Integration

The integration for Teradata enables the export of data from Treasure Data to your Teradata Server.

Prerequisites

  • Basic knowledge of Treasure Data
  • Basic knowledge of Teradata SQL
  • A Teradata instance running

Create a New Authentication

  1. Go to Integrations Hub > Catalog.
  2. Search and select Teradata.

  1. A dialog will open.

  1. Provide your configuration.
ParametersDescription
HostThe host information of the teradata database, eg. an IP address.
PortThe connection port on the instance, Teradata default is 1025.
UsernameUsername to connect to the teradata database.
PasswordPassword to connect to the teradata database.
JDBC Connection optionsAny special JDBC connections required by Teradata database. Please don't add these JDBC options because they will be ignored:
  • DBS_PORT (use port config)
  • DATABASE (use database config)
  • USER (use user config)
  • PASSWORD (use password config)
  • LOGMECH (currently support TD2 only)
connection timeoutTimeout (in seconds) for socket connection (default is 300).
Network timeoutTimeout (in seconds) for network socket operations (default is 1800). 0 means no timeout.
  1. Type a name for your connection and select Done.

Configure Export Results

In this step, you create or reuse a query and configure the data connection in the query.

Compatible export data between TD and Teradata

TD/Teradatastringdoublelongbooleantimestampjson
BYTEINTyesyesyesyesnono
SMALLINTyesyesyesyesnono
INTEGERyesyesyesyesnono
BIGINTyesyesyesyesyes (EpochMilli)no
FLOATyesyesyesyesnono
DECIMALyesyesyesyesnono
VARCHARyesyesyesyesyesyes
CHARyesyesyesyesyesyes
CLOByesyesyesyesyesyes
DATEyes (expected to be in format yyyy-MM-dd)noyes (expected to be EpochMilli)noyesno
TIME* yes (expected to be in format HH:mm:[ss])noyes (expected to be EpochMilli)noyesno
TIMESTAMPyes (expected to be in format yyyy-MM-ddTHH:mm:ssZ)noyes (expected to be EpochMilli)noyesno

When types are incompatible, NULL values will be set. Other Teradata data types are unsupported.

Data type mapping

Here is default mapping from the TD types to Teradata types in case the target table does not exist:

TD resultsTeradata
jsonVARCHAR (1000)
stringVARCHAR (500)
doubleFLOAT
longBIGINT
timestampTIMESTAMP
booleanBYTEINT

You might want to set the different type as the default. Data Type Mapping is used to explicitly set a specific type (for example, VARCHAR) to a specific column.

If you use Data Mapping, the following is true:

Only applies when creating a new target table. The syntax for data type mapping parameter is: col_name_1:VARCHAR; col_name2:BIGINT. The column name must match the export result field name.

Configure the Connection by Specifying the Parameters

Go to the TD Console. Go to Data Workbench > Queries. Access the query that you plan to use to export data.

Select Export Results, located at the top of your query editor.

The Choose Integration dialog opens. You have two options when selecting a connection to use to export the results:

  1. Use an existing connection Type the connection name in the search box to filter and select your connection.

    The dialog opens: Insert Mode

    Truncate Mode

    Merge Mode

  2. Create a new Teradata Connection. Select New Integration, and select Teradata from the Type dropdown. You see the following:

    Fill in Teradata information for your new connection. Then click Next.

Configuration Parameters

ParameterDescription
Database name (required)Database name
Target Table (required)Target table name
Table Type (required)Table Type. Only applies when creating a new table
Mode (required)Operation mode
Primary Index Field(s) (required if mode is merge)Primary Index Field(s) of target table
Data Type Mapping (Optional)Map a target table field to specify data type. Only applies when create new table
Batch Size (Optional)Size of a single batch insert
Skip Truncate If New Data Empty? (Optional)Skip the truncate operation if there is no new data to load

(Optional) Other configurations

  • You can create an activation in the Audience Studio to export segment data to this integration.
  • You can use Scheduled Job with Result Export to periodically upload data to a target destination.

(Optional) Export Integration Using the workflow

Within Treasure Workflow, you can specify the use of a data connector to export data.

Learn more at Using Workflows to Export Data with the TD Toolbelt.

For mode insert

_export:
   td:
      database: teradata

+teradata_export_task:
   td>: export.sql
   database: ${td.database}
   result_connection: new_created_teradata
   result_settings:
      type: teradata
      database: xxx
      table: xxxx
      table_type: set
      mode: insert
      column_options: col_1:VARCHAR(50), col_2:BIGINT
      batch_size: 16777216

For mode truncate

_export:
   td:
      database: teradata

+teradata_export_task:
   td>: export.sql
   database: ${td.database}
   result_connection: new_created_teradata
   result_settings:
      type: teradata
      database: xxx
      table: xxxx
      table_type: set
      mode: truncate
      column_options: col_1:VARCHAR(50), col_2:BIGINT
      batch_size: 16777216
      skip_truncate_if_empty: true

For mode merge

_export:
   td:
      database: teradata

+teradata_export_task:
   td>: export.sql
   database: ${td.database}
   result_connection: new_created_teradata
   result_settings:
      type: teradata
      database: xxx
      table: xxxx
      table_type: set
      mode: merge
      column_options: col_1:VARCHAR(50), col_2:BIGINT
      batch_size: 16777216
      merge_keys: col_1,col_2

(Optional) Export Integration Using the CLI

You can also use CLI (Toolbelt) to export results to Teradata.

You need to specify the information for export to your Account using the --result option of the td query command. For more information about the td query command, refer to this article.

The format of the option is JSON, and the general structure is as follows.

For mode insert

{
   "type": "teradata",
   "host": "xxxx",
   "port": 1025,
   "user": "xxx",
   "password": "xxx",
   "options": "{\"xxxx\": \"xxxx\"}",
   "connect_timeout": 30,
   "socket_timeout": 60,
   "database": "xxx",
   "table": "xxxx",
   "table_type": "set",
   "mode": "insert",
   "column_options": "col_1:VARCHAR(50), col_2:BIGINT",
   "batch_size": 16777216
}

For mode truncate

{
   "type": "teradata",
   "host": "xxxx",
   "port": 1025,
   "user": "xxx",
   "password": "xxx",
   "options": "{\"xxxx\": \"xxxx\"}",
   "connect_timeout": 30,
   "socket_timeout": 60,
   "database": "xxx",
   "table": "xxxx",
   "table_type": "set",
   "mode": "truncate",
   "column_options": "col_1:VARCHAR(50), col_2:BIGINT",
   "batch_size": 16777216,
   "skip_truncate_if_empty": true
}

For mode merge

{
   "type": "teradata",
   "host": "xxxx",
   "port": 1025,
   "user": "xxx",
   "password": "xxx",
   "options": "{\"xxxx\": \"xxxx\"}",
   "connect_timeout": 30,
   "socket_timeout": 60,
   "database": "xxx",
   "table": "xxxx",
   "table_type": "set",
   "mode": "merge",
   "column_options": "col_1:VARCHAR(50), col_2:BIGINT",
   "batch_size": 16777216,
   "merge_keys": "col_1,col_2"
}

Parameters

NameDescriptionTypeValueDefault ValueRequired
typeconnector typestringteradataN/AYes
hostTeradata server hoststringN/AN/AYes
portTeradata server portnumberN/A1025Yes
optionsTeradata JDBC optionstring (in hash object format)N/AN/ANo
connect_timeoutTeradata logon timeoutnumberN/A300No
socket_timeoutNetwork socket operations timeoutnumberN/A1800No
databaseTeradata databasestringN/AN/AYes
table_typeTable TypestringSupport values: set, multisetsetYes
tableTable NamestringN/AN/AYes
modeModestringSupport values: insert, truncate, mergeinsertYes
column_optionsMap a table field to specify data typestringN/AN/ANo
batch_sizeSize of a single batch insertnumberN/A16777216No
merge_keysPrimary Index Field(s) of target tablestringN/AN/AYes if mode merge
skip_truncate_if_emptySkip the truncate operation if there is no new data to loadbooleantrue/falsefalseNo