Treasure Workflow is based on the Treasure Data created Digdag open source project. The original source of Digdag information can be found at docs.digdag.io. Most, but not all, Digdag operators can be used as part of Treasure Workflow.

Operator

Type

Description

call>:

Control

Calls another workflow

require>:

Control

Depends on another workflow

loop>:

Control

Repeat tasks

for_each>:

Control

Repeat tasks for values

for_range>:

Control

Repeat tasks for a range

if>:

Control

Conditional execution

fail>:

Control

Changes the workflow status to failed

wait>:FunctionalSuspends execution and waits for the duration specified

Control Operators

call>:

call> operator calls another workflow.

This operator embeds another workflow as a subtask. This operator waits until all tasks of the workflow complete.

# workflow1.dig
+step1:
  call>: another_workflow.dig
+step2:
  call>: common/shared_workflow.dig


# another_workflow.dig
+another:
  sh>: ../scripts/my_script.sh

Options

require>:

require> operator requires the completion of another workflow. This operator is similar to call> operator, but this operator doesn’t start the other workflow if it’s already running or has done for the same session time of this workflow. If the workflow is running or newly started, this operator waits until it completes.

+step1: require>: another_workflow ```

+step2: sh>: tasks/step2.sh ```

Options

loop>:

loop> operator runs subtasks multiple times. It repeats tasks.

This operator exports ${i} variable for the subtasks. Its value begins from 0. For example, if count is 3, a task runs with i=0, i=1, and i=2.

+repeat:
  loop>: 7
  _do:
    +step1:
      echo>: ${moment(session_time).add(i, 'days')} is ${i} days later than ${session_date}
    +step2:
      echo>: ${moment(session_time).add(i, 'hours')} is ${i} hours later than ${session_local_time}.

Options

for_each>:

Repeat tasks for values.

for_each> operator runs subtasks multiple times using sets of variables.

(This operator is EXPERIMENTAL. Parameters are subject to change)

+repeat:
  for_each>:
    fruit: [apple, orange]
    verb: [eat, throw]
  _do:
    echo>: ${verb} ${fruit}
    # this will generate 4 tasks:
    #  +for-fruit=apple&verb=eat:
    #    echo>: eat apple
    #  +for-fruit=apple&verb=throw:
    #    echo>: throw apple
    #  +for-fruit=orange&verb=eat:
    #    echo>: eat orange
    #  +for-fruit=orange&verb=throw:
    #    echo>: throw orange

Options

for_range>:

Repeat tasks for a range. This operator is EXPERIMENTAL. Parameters are subject to change.

for_range> operator runs subtasks multiple times using sets of variables.

This operator exports ${range.from}, ${range.to}, and ${range.index} variables for the subtasks. Index begins from 0.


+repeat:
  for_range>:
    from: 10
    to: 50
    step: 10
  _do:
    echo>: processing from ${range.from} to ${range.to}.
    # this will generate 4 tasks:
    #  +range-from=10&verb=20:
    #    echo>: processing from 10 to 20.
    #  +range-from=20&verb=30:
    #    echo>: processing from 20 to 30.
    #  +range-from=30&verb=40:
    #    echo>: processing from 30 to 40.
    #  +range-from=40&verb=50:
    #    echo>: processing from 40 to 50.


if>:

Conditional execution. This operator is EXPERIMENTAL. Parameters are subject to change.

if> operator runs subtasks if true is given.

+run_if_param_is_true:
  if>: ${param}
  _do:
    echo>: ${param} == true

Options

fail>:

Changes the workflow status to failed.

fail> always fails and makes the workflow failed.

This operator is used with the if> operator to validate results of a previous task. Use it with _checkdirective so that a workflow fails when the validation doesn’t pass.

+fail_if_too_few:
  if>: ${count < 10}
  _do:
    fail>: count is less than 10!

Options

Functional Operators

wait>: 

Waits for a specific duration in the workflow.

This operator temporarily suspends the current execution and waits for the duration specified. The operation is resumed once the wait duration has passed.

+wait_10s:
  wait>: 10s


Options