Page tree
Skip to end of metadata
Go to start of metadata

If you are brand new to Treasure Workflow, Welcome! This topic covers the most basic information about how to create a workflow in the TD console and in the command-line interface. If you are more experienced, skip this topic and review the Reference information.

You can create Treasure Workflow in the TD console and from the command line (CLI). With this flexibility, it's possible to create workflows in your local environment and run the same workflows in Treasure Data's environment.

Before you get started, review the prerequisites and limitations of Treasure Workflow

Create a Simple Workflow in the TD Console

Follow these steps to create, edit, and run a simple workflow.

You must have a database and table in Treasure Data to complete this workflow.

1. Open the TD Console.
2. Navigate to Data Workbench > Workflows.
3. Select New Workflow.
4. Enter a workflow name. 
5. Keep the defaulted Project Name or select a different one.
6. Click your mouse in the Workflow Template field and select Simple Import and Saved Query.

7. Review the visual preview (on the left) and the code preview (on the right).
8. Select New Workflow.

Edit the Workflow

After you select New Workflow an editor displays where you can make changes to the code. Each code section includes instructions marked with a hashtag (#).

Treasure Workflow uses a YAML syntax. Pay close attention to the indentation and use of colons.

Identify the Database

Replace dbname.table with the name of your database.

Import Data

Treasure Data uses the td_load operator to load data from storage, other databases, or services. In this example, we’ll reference a data connector’s Unique ID.

1. In the TD Console, navigate to Integrations Hub > Sources.
2. Select a connector and then select the more menu (…).
3. Select Copy Unique ID.
4. Go back to Data Workbench > Workflows.
5. Select your workflow and then select Project Revisions.
6. Select the Revision link.
7. Select Edit Files.
8. Replace the td_load>: link with the Copy Unique ID value.
9. Select Save & Commit.

Run the Workflow

You can run a workflow right away or create a schedule.

For more information, see working with Treasure Workflows in the TD Console.

Create a Simple Workflow in the Command-Line Interface

Follow these steps to create, edit, and run a simple workflow.

Prerequisites

You must have an existing "mydirectory". For more information, see Creating a Directory for your Workflow Files

1. Open the terminal.
2. Open your workflow directory.
cd /users/<yourname>/mydirectory
3. List the existing items in your directory to verify files. 
ls
4. Create a workflow file. This file must have a .dig extension.
cat > my_workflow.dig
5. Type in the contents of your workflow. Here's a simple example:
_export:
  tmp_var: hello world
 
+echo_task1:
  echo>: this is test task!
 
+echo_task2:
  echo>: ${tmp_var}
6. To stop editing, type Ctrl + D.
7. Create the project name and push the workflow to Treasure Data.
td wf push my_project
8. After you have created the project file in Treasure Data, your terminal displays the following:




  • No labels