Why do I get an unknown error when a loop runs many times?

The number of tasks registered in the workflow queue is limited to 1,000. In a loop, tasks are registered in the queue by subtask. To avoid this error, divide the tasks into smaller loops.

Can I send aggregated workflow results by email?

You cannot attach files via email; however, you can export partial results into the body of an email.

Can I rename a workflow?

No. An alternative is to delete the workflow and recreate it with a different name.

Is there a limit to the number of dependent workflows in a parent workflow?

Because the TD console provides visibility into all dependent workflows associated with a parent workflow, you should try to limit the dependent workflows to less than 40.

Can I automatically stop a workflow schedule?

Yes. Workflows with a start date and end date provide flexible scheduling capabilities and eliminate wasted computer resources for any unnecessary dummy workflow running.

Why isn't the daily workflow that I scheduled for today running?

When Treasure Data receives a request to run a workflow, it automatically creates a session, an advanced plan to run the workflow, in contrast to an attempt, which is the actual attempt to run the workflow at a scheduled time.

Scenario

The following scenario explains why a workflow you expect to run today will run the next day.

You authored a new workflow today and ran it as a test. After you are satisfied with the results, you want to set run the workflow to run on a daily schedule, starting from today.

The current time is April 27, 2022, 14:20:10.

You create a scheduled workflow to run daily starting from today.

_export:
  td:
    database: jean_wf_temp

timezone: America/Los_Angeles

schedule:

   start: 2202-04-27
   daily>: 22:35:00

   end: 2022-05-30

+setup:

  sh>: echo ${session_time}

After you save the workflow, check the Next Attempt time. In this scenario, you see the following:

Actual: “Apr 28, 2022 10:35 pm”

Expected: “Apr 27, 2022 10:35 pm”

The workflow session already exists as it was run earlier today for testing, hence, the next acceptable session time for the workflow is 2022:04:28 00:00:00 and the scheduled run time is 2022:04:28 22:35:00, i.e. 10:35 pm on April 28, 2022.

Notes

  • The session time is the plan to run the workflow, while the scheduled time is the time when the attempt to run the workflow is made.
  • For daily scheduled workflows, the session time is the run day's zeroth hour, i.e. YYYY:MM:DD 00:00:00.
  • For hourly scheduled workflows, the session time is the hour's zeroth minute, i.e. HH:00:00.

Example

If the system clock at this time is: 2022-04-27 14:20:10 +0900.

Schedule

First session time

First scheduled to run at

hourly>: “32:32”

2022-04-27 14:00:00 +0900

2022-04-27 14:32:32 +0900

daily>: “10:32:32”

2022-04-28 00:00:00 +0900

2022-04-28 10:32:32 +0900

weekly>: “2,10:32:32”

2022-04-29 00:00:00 +0900

2022-04-29 10:32:32 +090

monthly>: “2,10:32:32”

2022-05-02 00:00:00 +0900

2022-05-02 10:32:32 +0900

Why did my workflow end before 12 pm on the end date?

When you create a workflow, you can change the default timezone value of UTC to another timezone. If you do not change the timezone, and set the workflow schedule to run hourly, the schedule run is based on the system timezone, i.e. UTC.

For example, let's assume you are in the Asia/Tokyo timezone, and you schedule an hourly workflow with an end date of 2022-05-30. Unfortunately, you didn't change the timezone for the workflow, so it's defaulted to UTC. That means that the last scheduled run of the workflow could be around 4:20 pm on May 30th, 2022, which is 9 hours earlier (UTC time difference) than the anticipated last run of 11:20 pm on May 30th 2022. 

schedule:
  start: 2022-05-27
  hourly>: 20:00
  end: 2022-05-30

+setup:
  echo>: start ${session_time}














  • No labels