Release Note 20180301

This is a summary of new features and improvements introduced in the March 1st, 2018 release. If you have any product feature requests, submit them at feedback.treasuredata.com.

Table of Contents

Output: Result Output to Amplitude

Now you can send Events data from Treasure Data into Amplitude.

Console: Workflow UI

We are releasing our new version of Workflow UI for General Availability! With the new Treasure Workflow UI, you can now create and manage complex data pipelines entirely from the Treasure Data web console. Data pipelines can significantly help in data analysts' and engineers' work to developing a number of common data processing flows, including:

  • Coordinated Input and Output Data Transfers
  • Data Cleanup and Enrichment Steps
  • Building a unified customer data model for your company
  • Managing processing steps, both on and off Treasure Data

This Workflow UI GA does not represent a change to our core workflow system. Instead, the user interface improves your experience. No migration is required for users and workflows still operate the same way.

This release will happen in two phases.

Phase 1: For Current Workflow Users, on March 1st Phase 2: For All Customers, on March 27th

Check out our notes for more information on what is changing and what you can expect from Treasure Workflow.

Console: Result Export Job Separation from Query Job (coming soon)

In March, we will migrate all accounts to an updated job processing framework, which separates the internal elements of Query Execution & Result Export.

This work represents a major milestone towards a customer-facing separation of Query & Data export processing. When we complete this future work, you will be able to retry data exports, without having to also retry the query processing itself.

With this system migration, you might notice the following changes in the behavior of your account:

  1. You might notice an increase in query concurrency. When a job reaches the result export phase, the query engine will now allow for another query to be submitted if there are resources available within your Presto or Hive resource limits.

  2. You will notice that for every query with a Result Export setup, two jobs will now be generated. As a result, you will see the following in your jobs list for queries with result export setup:



  3. Full logs will now be split between the two jobs.

In the Presto/Hive Query Job, the end of the log will now indicate the new Result Export job being created:



In the Result Export Job, you can see the related export jobs that you used to see within the Query Job:



Both jobs will fail if the result export fails. If the query fails, the result export will not be generated.



Finally, the query job will not show as completed, until the result export job is also completed.


Last modified: Mar 01 2018 19:32:43 UTC

If this article is incorrect or outdated, or omits critical information, let us know. For all other issues, access our support channels.