Learn more about Microsoft Azure Blob Storage Import Integration.

This article explains how to write job results directly to your Microsoft Azure Blob Storage.

For sample workflows of exporting to Microsoft Azure Blob Storage, view Treasure Boxes.

Prerequisites

  • Basic knowledge of Treasure Data, including the TD Toolbelt.

  • A Microsoft Azure Platform account

Create Azure Storage Account

First, you need to create an Azure Storage account. Here’s documentation about how to create it.

In the Azure Portal, navigate to your storage account, select All Settings, and then select Access keys to obtain account access keys.

Usage

Create the Destination Container in Microsoft Azure Blob Storage

Create your container of Azure Blob Storage from your Azure portal.

Also, check the Naming and Referencing Containers, Blobs, and Metadata roles.

Write the Query

Visit the TD Console query editor page and compile your query.

Specify the Result Export target

In the same window, select the Add button in the Result Export section and select Microsoft Azure Blob Storage from the drop-down menu. Edit all of your credentials information, including your Account Name, Account Key, Container, and Path.

Configure Export Results in Your Data Connection

  1. Navigate to Data Workbench > Queries.

  2. Select an existing query or create a new one.

  3. In the query editor, select the Export Results checkbox.

  4. Select your Microsoft Azure connection
  5. The Export Results dialog pane opens. Edit the following parameters.

    Parameters

    Description

    Container (container)

    Azure blob container name

    Path prefix (path_prefix)

    Files will be stored at this path

    Blob Type (blob_type)UNSPECIFIED or BLOCK_BLOB. Specify BLOCK_BLOB if your result is very large
    Format (format)The output file format (csv or tsv)
    Compression (compression)Compress output file. Supported compressions: Gzip or Bzip2 (gz / bz2)
    Delimiter (delimiter)Delimiter character such as "," for CSV, "\t" for TSV, "|" or any single-byte character
    Null string (null_string)Use this value to represent NULL values. Defaults to an empty string
    End-of-line character (newline)End of line character of the result file

Execute the query

Either save the query with a name and run or just run the query. Upon successful completion of the query, the results are automatically imported to the specified container destination.

Usage from CLI

You can also use CLI for Result Export to Microsoft Azure Blob Storage.

account_name/account_key case

$ td query --result '{"type":"azure_blob_storage","account_name":"xxxx","account_key":"xxxx","container":"xxxx","path_prefix":"/path/to/file.csv","sequence_format":"","header_line":true,"quote_policy":"MINIMAL","delimiter":",","null_string":"","newline":"CRLF"}' -d sample_datasets "select * from www_access" -T presto


Optional Configure Export Results in Workflow

Within Treasure Workflow, you can specify the use of this data connector to output data.

timezone: UTC
 
_export:
	td:
		database: sample_datasets
 
+td-result-into-target:
	td>: queries/sample.sql
	result_connection: my_azure_blob
	result_settings:
		container: test_container
		path_prefix: /path/to/file.csv
		format: csv
		blob_type: BLOCK_BLOB
	


Learn more about using data connectors to export data in workflows using CLI.


  • No labels