Release Note 20170101

Happy New Year! This is a summary of new features and improvements introduced in the January 1, 2017 release. If you have any product feature request, please file it at feedback.treasuredata.com.

Table of Contents

Treasure BI: Lightweight BI Add-on

Treasure BI (Business Intelligence) is a lightweight cloud BI option for Treasure Data, for small to medium size customers. You can create reports and dashboards and deliver them to your entire team.

This feature is available as an Add-On to our core cloud service offering. Please contact your Treasure Data representative for the trial.

Collection: JavaScript SDK with Device Fingerprint

Device Fingerprint is a commonly used techniques to create hash value from device information to create unique value not depending on the 3rd party cookie. We have published an article about how to use fingerprintjs2 library with JavaScript SDK.

// Enable device fingerprint
new Fingerprint2().get(function (result) {
  td.set('$global', 'td_fingerprint_id', result)
});

Please refer to the documentation below.

Workflow: The Workflow Console Now Shows Task Execution Timeline

The ability to see the relative duration of individual tasks within a workflow will allow users to focus on the tasks that take the longest when optimizing their workflow.

Workflow: Running EMR Jobs (beta)

Many customers want to run analysis built in code against data stored in tables on Treasure Data. Now you can schedule using the new EMR operator using Treasure Workflow. These EMR jobs will run in your AWS environment, but clusters & jobs will the started & stoped using configurations. Now you can build unified workflows that combined queries operating in Treasure Data with EMR jobs.

Please refer to the documentation below.

Workflow: Handle Frequently Running Workflows that Sometimes Take Too Long

Sometimes you have frequently running workflows (e.g. sessions every 30 or 60 minutes) that take longer than the duration between sessions. This variability in the duration of a workflow can occur for a number reasons. For example, you may be seeing an increase in the amount of data you are normally processing.

For example, let’s say we have a workflow that is running hourly, and it normally takes only 30 minutes. But it’s the holiday season and now there has been a huge increase in usage of your site – so much data is now being process the workflow is taking 1 hour and 30 minutes. During this time period, a 2nd workflow has started running for the following hour, which causes further strain on your available resources because both are running at the same time.

It’s this case it’s best to skip the next hour’s workflow session, and instead utilize the subsequent session to process 2 hours of data. To do this, we’ve added the following:

  • Added a skip_on_overtime: true | false schedule option that can be used to control whether scheduled session execution should be skipped if another session is already running.
  • Scheduled workflow sessions now have a last_executed_session_time variable which contains the previously executed session time. It is usually same with last_session_time but has different value when skip_on_overtime: true is set or the session is the first execution.

Workflow: Added rename_table option to td_ddl operator

Now you can change the name of a table during the execution of a workflow.

Please refer to the documentation below.

Collection: td-agent GPG key update

Starting on 2017/01/01, the major Linux packaging system apt has shut off support for SHA1 encrypted GPG key for package signing. This is because SHA1 algorithm is considered weak nowadays. Thus we have upgraded td-agent’s GPG key with SHA2 encrypted key. Please upgrade your key by following the instruction described here.

Collection: td-agent v2.3.4

td-agent v2.3.4 was released. This includes numerous bug fixes, performance improvements, and new features. Everyone is recommended to upgrade.

Client: Python Client Library v0.8.0

td-client-python v0.8.0 was released. This includes a couple of bug fixes and new improvements including unicode encoding issues on Python 2.x.


Last modified: Jan 06 2017 19:21:58 UTC

If this article is incorrect or outdated, or omits critical information, please let us know. For all other issues, please see our support channels.