Visit our new documentation site! This documentation page is no longer updated.

Writing Job Results to a SFTP Server

You can send job results directly to your SFTP server.

Table of Contents

Prerequisites

  • Basic knowledge of Treasure Data, including the toolbelt
  • An sftp server

Basic Usage



Untitled-3
You can limit the access to your endpoint by using a list of static IPs. Contact support@treasuredata.com if you need it.

Get your credential for your SFTP server

First, obtain your credential to access your SFTP server.

Specify the Result Export target

Go to the Treasure Data Console query editor page, click ’New Query.’

Click Output Results and select SFTP from the drop down menu. Fill out all the information, including your Host, Port, Username, Private Keyfile, and Passphrase of keyfile (if available).



Rename remote file after upload

If you select the “Rename file after upload finish” option, SFTP result output renames the file on the remote SFTP server from “.xxx.tmp” to “.xxx” after all the data is transferred. Some MA tools will try to import data when a file with specific name exists on SFTP server. The temp name option is useful for such cases.

Execute the query

Finally, either save the query with a name and run, or just run the query. Upon successful completion of the query, the query result is automatically imported to the specified Container destination.

Usage from CLI

You can also use CLI for Result Export to SFTP.

secret_key_file case

Untitled-3
In the following example, note that the escape newline with backslash in the key.

Example,

$ td query --result '{"type":"sftp","host":"xx.xx.xx.xx","port":22,"username":"xxxx","secret_key_file":"{\"content\":\"-----BEGIN RSA PRIVATE KEY-----\nABCDEFJ\nABCDEFJ\n-----END RSA PRIVATE KEY-----\"}","secret_key_passphrase":"xxxxxx", "user_directory_is_root":true,"path_prefix":"/path/to/file.csv","sequence_format":"","rename_file_after_upload":false,"header_line":true,"quote_policy":"MINIMAL","delimiter":",","null_string":"","newline":"CRLF"}' -d sample_datasets "select * from www_access" -T presto

user/password case

Example,

$ td query --result '{"type":"sftp","host":"xx.xx.xx.xx","port":22,"auth_method":"Password","username":"xxxx","password":"xxxxx","user_directory_is_root":true,"path_prefix":"/path/to/file.csv","sequence_format":"","rename_file_after_upload":false,"header_line":true,"quote_policy":"MINIMAL","delimiter":",","null_string":"","newline":"CRLF"}' -d sample_datasets "select * from www_access" -T presto

Last modified: Mar 22 2018 18:34:41 UTC

If this article is incorrect or outdated, or omits critical information, let us know. For all other issues, access our support channels.