Page tree
Skip to end of metadata
Go to start of metadata

You can parallelize the query result output process by using the CREATE TABLE AS SELECT statement. If you DROP the table before running the query, your performance is significantly better. The result output performance will be 5x faster than running SELECT *. Treasure Data Presto skips the JSON output process and directly produces a 1-hour partitioned table.

Without using DROP TABLE, Presto uses JSON text to materialize query results. And if the result table contains 100GB of data, the coordinator transfers more than 100GB of JSON text to save the query result. So, even if the query computation is almost finished, output of theJSON results takes a long time.

To clean up the result table beforehand:

  1. Add a DROP TABLE statement at the top of your query.

  2. Use CREATE TABLE (table) AS SELECT … 

For example, your query might look like this:

DROP TABLE IF EXISTS my_result;
CREATE TABLE my_result AS 
    SELECT * FROM my_table;

  • No labels