For On-Demand Jobs

For on-demand jobs, just add the --result / -r option to the td query command. After the job is finished, the results are written into your collection.

$ td query --result 'mongodb://user:password@host:1234/database/collection' \
-w -d testdb "SELECT code, COUNT(1) FROM www_access GROUP BY code"

For Scheduled Jobs

For scheduled jobs, add the --result / -r option when scheduling a job. Every time the job runs, the results are written into mytbl.

$ td result:create mydb 'mongodb://user:password@host:1234/database'
$ td sched:create hourly_count_example "0 * * * *" \
-d testdb "select count(*) from www_access" --result mydb:mycollection

Format

The result output target is represented by a URL with the following format:

mongodb://<username>:<password>@<host>/<database>/<collection>
mongodb://<username>:<password>@<host>:<port>/<database>/<collection>

where:

  • mongodb is identified for result output to MongoDB;

  • username and password are the credential to the MongoDB instance;

  • hostname is the host name of the MongoDB instance;

  • port is the port number through which the MongoDB instance is accessible. This is optional;

  • database is the name of the destination database;

  • collection is the name of the destination collection.

Modes

You can add or delete data using the following modes:

mongodb://user:password@host:1234/database/collection # append
mongodb://user:password@host:1234/database/collection?mode=append # append
mongodb://user:password@host:1234/database/collection?mode=replace # replace
mongodb://user:password@host:1234/database/collection?mode=truncate # truncate
mongodb://user:password@host:1234/database/collection?mode=update&unique=key1 # update

mode=append (default)

This is the default mode. The query results are appended to a collection. If the collection does not exist yet, a new collection is created.

This method is atomic.

mode=replace

If the collection already exists, the rows of the existing collection are replaced with the query results. If the collection does not exist yet, a new collection is created.

We achieve atomicity (so that a consumer of the collection always has consistent data) by performing the following three steps in a single transaction.

  1. Create a temporary collection.

  2. Write to the temporary collection.

  3. Replace the existing collection with the temporary collection using RENAME command.

This method is atomic.

mode=truncate

The system first truncates the existing collection, then inserts the query results. If the collection does not exist yet, a new collection is created.

Unlike REPLACE, TRUNCATE retains the indexes of your collection.

This method is atomic.

mode=update

This mode uses MongoDB’s find and “upsert” method (see MongoDB’s documentation). In short, a row is inserted unless it would cause a duplicate value in the unique index or primary key, in which case an update is performed. Make sure you’ve already created unique index on the fields you specified at the arguments. When this mode is used, the unique option is required.

Because MongoDB doesn’t support transactions, this mode cannot guarantee transaction atomicity.

Unique

This option is only relevant and required with the update mode. It takes the name of the unique key or keys (command separated) column name to use for updating the MongoDB collection.

Example

mongodb://user:password@host:1234/database/collection?mode=update&unique=key1
mongodb://user:password@host:1234/database/collection?mode=update&unique=key1,key2,key3
  • No labels