Learn more about KARTE Import Integration.

You can use the Karte connector to write job results from queries run in Treasure Data to your Google Cloud Storage.

Prerequisites

• Basic knowledge of Treasure Data, including the Toolbelt.

• A Google Cloud Platform account

Getting Google Cloud Platform Credentials

Before creating the connector, you need to gather the following information: your Project ID, and JSON Credential.

JSON Credential

The integration with Google Cloud Storage is based on server-to-server API authentication. Go to your Google Developer Console and select Credentials under APIs & auth at the left menu. Then, select Service account.

Select the JSON based key type, which is Google’s recommended configuration. The key is automatically downloaded by the browser.

Specifying Output in Your KARTE Connector

Using the TD Console

Create the destination Bucket in Google Cloud Storage

Create your bucket from your Google Cloud Storage console.

Write the query

Go to the Treasure Data Console query editor page and compile your query.

Specify the Result Export target

In the same window, select the Create New Connection in the Result Export.

In the dialog opens, search for and select your connection type: KARTE. Provide a name for your KARTE connection. Complete the information, including your JSON Credential, Bucket name, and Path:

Execute the query

Finally, either save the query with a name or just run the query. After the successful completion of the query, the results are automatically sent to the specified Google Cloud Storage destination.

Use the CLI

You can set a scheduled query to send job results to Google Cloud Storage using KARTE.

Example

$ td sched:create scheduled_karte "10 6 * * *" \
-d dataconnector_db "SELECT id,account,purchase,comment,time FROM data_connectors" \
-r '{"type":"karte","bucket":"samplebucket","path_prefix":"/output/test.csv","format":"csv","compression":"","header_line":false,"delimiter":",","null_string":"","newline":"CRLF",  "json_keyfile":"{\"private_key_id\": \"ABCDEFGHIJ\", \"private_key\": \"-----BEGIN PRIVATE KEY-----\\nABCDEFGHIJ\\ABCDEFGHIJ\\n-----END PRIVATE KEY-----\\n\", \"client_email\": \"ABCDEFGHIJ@developer.gserviceaccount.com\", \"client_id\": \"ABCDEFGHIJ.apps.googleusercontent.com\", \"type\": \"service_account\"}"}'

(Optional) Configure Export Results in Workflow

Within Treasure Workflow, you can specify the use of this data connector to output data.

timezone: UTC

_export:
  td:
    database: sample_datasets

+td-result-into-target:
  td>: queries/sample.sql
  result_connection: karte_integration
  result_settings:
    format: csv
    newline: CR
    ....

For more information on using data connectors in Treasure Data Workflow to export data, review About Using Workflows to Export Data with TD Toolbelt.

KARTE Connector Parameters

You can use the following parameters when configuring the KARTE connector.