Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Reverted from v. 26

...

*If quote_policy is NONE, the `quote` option is ignored and \ is used as the default `escape` option ← need check. The quote_policy option is used to determine the field type to quote as follows.

Sample Workflow Code

For samples of workflows, see Treasure Boxes.

Excerpt


Numbered Headings
start-numbering-with1
start-numbering-ath5
Navigate to Data Workbench > Queries.
Create a new query or select an existing query.
Next to Schedule, select None.

In the drop-down, select one of the following schedule options.

Drop-down ValueDescription
Custom cron...

Review Custom cron... details.

@daily (midnight)Run once a day at midnight (00:00 am) in the specified time zone.
@hourly (:00)Run every hour at 00 minutes.
NoneNo schedule.

Custom cron... Details

Cron Value

Description

0 * * * *

Run once an hour

0 0 * * *

Run once a day at midnight

0 0 1 * *

Run once a month at midnight on the morning of the first day of the month

""

Create a job that has no scheduled run time.


Code Block
linenumberstrue
 *    *    *    *    *
 -    -    -    -    -
 |    |    |    |    |
 |    |    |    |    +----- day of week (0 - 6) (Sunday=0)
 |    |    |    +---------- month (1 - 12)
 |    |    +--------------- day of month (1 - 31)
 |    +-------------------- hour (0 - 23)
 +------------------------- min (0 - 59)

The following named entries can be used:

  • Day of Week: sun, mon, tue, wed, thu, fri, sat

  • Month: jan, feb, mar, apr, may, jun, jul, aug, sep, oct, nov, dec

A single space is required between each field. The values for each field can be composed of:

Field ValueExampleExample Description

a single value, within the limits displayed above for each field.



a wildcard ‘*’ to indicate no restriction based on the field. 

‘0 0 1 * *’ configures the schedule to run at midnight (00:00) on the first day of each month.
a range ‘2-5’, indicating the range of accepted values for the field.‘0 0 1-10 * *’ configures the schedule to run at midnight (00:00) on the first 10 days of each month.
a list of comma-separated values ‘2,3,4,5’, indicating the list of accepted values for the field.

0 0 1,11,21 * *’


configures the schedule to run at midnight (00:00) every 1st, 11th, and 21st day of each month.
a periodicity indicator ‘*/5’ to express how often based on the field’s valid range of values a schedule is allowed to run.

‘30 */2 1 * *’


configures the schedule to run on the 1st of every month, every 2 hours starting at 00:30. ‘0 0 */5 * *’ configures the schedule to run at midnight (00:00) every 5 days starting on the 5th of each month.
a comma-separated list of any of the above except the ‘*’ wildcard is also supported ‘2,*/5,8-10’‘0 0 5,*/10,25 * *’configures the schedule to run at midnight (00:00) every 5th, 10th, 20th, and 25th day of each month.
 (Optional) If you enabled the Delay execution, you can delay the start time of a query.

Execute the Query

Save the query with a name and run, or just run the query. Upon successful completion of the query, the query result is automatically imported to the specified container destination.

Export to SFTP Server Using Workflow

You can export the result of a job to SFTP by using td>: operator of workflow. If you add `result_connection` and `result_settings` options, the task exports the result of the job of the run query.

Learn more at Using Workflows to Export Data with the TD Toolbelt.

Code Block
linenumberstrue
+export_sftp:
  td>: example.sql
  result_connection: your_authentication_name
  result_settings:
	user_directory_is_root: true
    path_prefix: /test/20210510.csv
    format: csv
    header_line: true

Parameters

NameDescriptionValueDefault ValueRequired

user_directory_is_root

Treat the user directory as the root directory

true or false

false

No

path_prefix

Files are stored at this path.

N/A

N/A

Yes

rename_file_after_upload

Renames the file on the remote SFTP server from “.xxx.tmp” to “.xxx” after all the data is transferred. Some MA tools try to import data when a file with a specific name exists on the SFTP server. The temp name option is useful for such cases.

true or false

false

No

format

Format of the exported files. You can choose CSV or TSV.

  • CSV
  • TSV

CSV

No

compression

Compression format of the exported file.

  • None
  • gz
  • bzip2

None

No

header_line

If true, write the header line with column name at the first line.

true or false

true

No

delimiter

Delimiter character

  • default(, for CSV, \t for TSV)
  • ,
  • \t
  • Tab
  • |
  • Any single-byte character

default

No

quote_policy*

Policy for quote

  • ALL
  • MINIMAL
  • NONE

MINIMAL for CSV,

NONE for TSV

No

null_string

Expression of NULL values

empty string for CSV

\N for TSV

No

newline

Newline character

  • CRLF
  • CR
    LF

CRLF

No

temp_file_threshold

The maximum file size (in bytes) of local temp file. When the temp file reaches the threshold, the file is flushed to a remote file.

If you encounter an error like `channel is broken`, reducing the value of this option might resolve the error.

5368709120

No

NameDescription

ALL

Quote all fields.

MINIMAL

Only quote those fields which contain a delimiter, quote, or any of the characters in lineterminator.

NONE

Never quote fields. When the delimiter occurs in field, escape with escape character