Data Landing is an intermediate storage area within Treasure Data ecosystem. This intermediate storage area provides the capacity for compliance with corporate security policies that you might have in place, and helps to avoid extra efforts in procuring intermediate storage solutions when integrating your data with Treasure Data. Because some corporate policies forbid directly sending files with Data Connectors from/to external cloud storage, an intermediate storage area can be used to stage and then import files into Treasure Data CDP or export to Data Landing for import back into your company's internal storage.

The data landing integration provides seamless access to an intermediate storage area so that data files can be imported into TD. The data landing intermediate storage area might also allow you to transfer files from various software and systems that do not otherwise have TD integrations.

Having access to an intermediate storage area that requires minimal setup also provides a short-term storage and retrieval area for machine learning models (binaries) that can be created with custom scripts.

You can 

  • access intermediate storage with your SFTP client or any other compatible tool
  • transfer data between intermediate storage and TD with Data Connectors


Prerequisites

  • Basic Knowledge of Treasure Data CDP

  • SFTP client
  • Data Landing user account

This feature is in beta. Contact your Customer Success representative for more information.

Limitations

  • Only one Data Landing account with read/write access can be created per CDP customer can be created.
  • Data in the Data Landing area is retained for 2 weeks after it was created or last modified. After 14 days, it is a subject for removal, but may take another day or two for removal to occur.
  • About 500Gb of storage is allowed per account.
  • Premium Audit Log is not supported in Data Landing.

Creating Account

You will have to contact Support or Customer Success to create a user account and get a user id issued. When you contact them, have
an OpenSSH public key (at least 2048-bit long, RSA format, PEM) ready. A passphrase can be used but is not required.

Optionally, you can provide a source IP range CIDR to allowlist the access to your data only from IPs falling within the specified range.

To create an SSH key, open a command terminal and enter:

ssh-keygen -t rsa -P "" -m PEM -f <key_name>

This will generate a private/public key pair.

Data Transfer

Data Landing service supports only standard port 22.

Supported commands:

  • cd
  • ls/dir
  • pwd
  • put
  • get
  • rename (only for file names)
  • mkdir
  • rm/delete
  • rmdir (for non-empty folders only).

Upload/Download to/from Data Landing

To log in from a command terminal, enter:

sftp -i <key-name> <issued_user_id>@<service_endpoint>

Upload your file:

sftp> put test.csv

Uploading test.csv to /test.csv

test.csv


From a UI client, enter the endpoint, username (issued user id), password (if used) and specify your private key location.

Example for Cyberduck:

Press Connect, and access your SFTP home directory

Service Endpoints

You will be notified about the endpoint when you get your account created.

Recommended Clients

Working with Data Connectors

You can use Data Connectors for import to CDP and export from CDP in the same manner as described in SFTP Server Import Integration and SFTP Server Export Integration.


  • No labels