Skip to content
/
Microsoft Power Bi Export...
Last updated

Microsoft Power Bi Export Integration

Power BI is an analytics tool used to analyze data and share insights. You can write job results directly into your Microsoft Power BI datasets.

Limitations

Limitations are controlled by Microsoft and are subject to change. See Power BI Rest API Limitations.

  • 75 max columns
  • 75 max tables
  • 10,000 max rows per single POST rows request
  • 1,000,000 rows added per hour per dataset
  • 5 max pending POST rows requests per dataset
  • 120 POST rows requests per minute per dataset
  • If a table has 250,000 or more rows, 120 POST rows requests per hour per dataset
  • 200,000 max rows stored per table in FIFO dataset
  • 5,000,000 max rows stored per table in ‘none retention policy’ dataset
  • 4,000 characters per value for string column in POST rows operation
  • Only one table can be exported into one dataset.
  • Regarding Power BI dataset types, only Push Dataset can be used. (See Datasets in the Power BI service.)

Prerequisites

  • Basic knowledge of Treasure Data, including the TD Toolbelt.
  • A Microsoft Power BI account as an organizational user for Azure Active Directory tenant who can grant permissions to Treasure Data.
  • For the OAuth authentication, you need to give approval to yourself, because this integration does not support an authentication flow where the OneDrive user can obtain approval from an administrator.

Creating the Export Dataset

  1. Open TD Console.

  2. Navigate to the Data Workbench > Queries.

  3. Select New Query.

  4. Select the database for which you would like to export data.

  5. Type your query into the query editor.

  6. Select Output Results.

    A dialog Choose Saved Connection is displayed.

  7. Select an existing power_bi connection.

    If you do not already have a Saved Connection, see Creating a New Connection During Data Export.

  8. Specify the export details. A dataset and table are required.

  1. Optionally, type a value for Workspace ID.

    The Workspace ID is a PowerBI or group ID allowing users to share their work.

  2. Optionally, choose whether to replace existing data or not.

If chosen, TD deletes all the data before replacing it with fresh data during the export.

  1. Review the limits for your plan and type a value for your Service plan row limit.

Max rows are added per hour per dataset for service plans. The free service plan allows 10,000 rows. The paid service plan allows 1,000,000 rows. 12. Select Done.

Creating a New Connection During Data Export

  1. From the Choose Integration screen, select New Integration.

  2. Select Microsoft Power BI.

  3. Select Catalog.

  4. Search and select Power BI. The dialog opens:

  5. Select the link under OAuth connection to create a new one. You can also select an existing connection and skip the rest of this section.

  6. Sign in to your Power BI account from the popup window:

    You will get a list of permissions to grant access to.

  7. Select Accept.

    You are redirected back to Treasure Data Connections.

  8. Select the OAuth connection that you just created.

  9. Name and finish creating the connection.

You can use the power_bi connection in Sources.

Example Query and Export

Here is an example of a Power BI connection with an exporting column with the timestamp format on Treasure Data. You would run the following query with PowerBI as Result Export:

SELECT Category, Name, parse_datetime(created_date, 'yyyy-MM-DD HH:mm:ss.SSS') as ManufacturedOn, isCompete, ProductID FROM (
  VALUES ('Cars', 'Suzuki', '2015-01-16 07:15:33.123', true, 15),
         ('Bikes', 'Silver', '2016-05-20 13:52:44.673', false, 17),
         ('Cars', 'Toyota', '2017-09-27 22:32:34.456', false, 20)
)  tbl (Category, Name, created_date, isCompete, ProductID)

This example query requires no source table.

You must choose your database, pick “sample_datasets”.

Pick Presto as the query engine.

The query should be completed in a few seconds.

After successfully completing the query, the result is automatically imported to the specified database destination.

Check the dataset in PowerBI’s dashboard and verify that the data was exported:

(Optional) Schedule Query Export Jobs

You can use Scheduled Jobs with Result Export to periodically write the output result to a target destination that you specify.

Treasure Data's scheduler feature supports periodic query execution to achieve high availability.

When two specifications provide conflicting schedule specifications, the specification requesting to execute more often is followed while the other schedule specification is ignored.

For example, if the cron schedule is '0 0 1 * 1', then the 'day of month' specification and 'day of week' are discordant because the former specification requires it to run every first day of each month at midnight (00:00), while the latter specification requires it to run every Monday at midnight (00:00). The latter specification is followed.

Scheduling your Job Using TD Console

  1. Navigate to Data Workbench > Queries

  2. Create a new query or select an existing query.

  3. Next to Schedule, select None.

  4. In the drop-down, select one of the following schedule options:

    Drop-down ValueDescription
    Custom cron...Review Custom cron... details.
    @daily (midnight)Run once a day at midnight (00:00 am) in the specified time zone.
    @hourly (:00)Run every hour at 00 minutes.
    NoneNo schedule.

Custom cron... Details

Cron ValueDescription
0 * * * *Run once an hour.
0 0 * * *Run once a day at midnight.
0 0 1 * *Run once a month at midnight on the morning of the first day of the month.
""Create a job that has no scheduled run time.
 *    *    *    *    *
 -    -    -    -    -
 |    |    |    |    |
 |    |    |    |    +----- day of week (0 - 6) (Sunday=0)
 |    |    |    +---------- month (1 - 12)
 |    |    +--------------- day of month (1 - 31)
 |    +-------------------- hour (0 - 23)
 +------------------------- min (0 - 59)

The following named entries can be used:

  • Day of Week: sun, mon, tue, wed, thu, fri, sat.
  • Month: jan, feb, mar, apr, may, jun, jul, aug, sep, oct, nov, dec.

A single space is required between each field. The values for each field can be composed of:

Field ValueExampleExample Description
A single value, within the limits displayed above for each field.
A wildcard '*' to indicate no restriction based on the field.'0 0 1 * *'Configures the schedule to run at midnight (00:00) on the first day of each month.
A range '2-5', indicating the range of accepted values for the field.'0 0 1-10 * *'Configures the schedule to run at midnight (00:00) on the first 10 days of each month.
A list of comma-separated values '2,3,4,5', indicating the list of accepted values for the field.0 0 1,11,21 * *'Configures the schedule to run at midnight (00:00) every 1st, 11th, and 21st day of each month.
A periodicity indicator '*/5' to express how often based on the field's valid range of values a schedule is allowed to run.'30 */2 1 * *'Configures the schedule to run on the 1st of every month, every 2 hours starting at 00:30. '0 0 */5 * *' configures the schedule to run at midnight (00:00) every 5 days starting on the 5th of each month.
A comma-separated list of any of the above except the '*' wildcard is also supported '2,*/5,8-10'.'0 0 5,*/10,25 * *'Configures the schedule to run at midnight (00:00) every 5th, 10th, 20th, and 25th day of each month.
  1. (Optional) You can delay the start time of a query by enabling the Delay execution.

Execute the Query

Save the query with a name and run, or just run the query. Upon successful completion of the query, the query result is automatically exported to the specified destination.

Scheduled jobs that continuously fail due to configuration errors may be disabled on the system side after several notifications.

(Optional) You can delay the start time of a query by enabling the Delay execution.

Activate a Segment in Audience Studio

You can also send segment data to the target platform by creating an activation in the Audience Studio.

  1. Navigate to Audience Studio.
  2. Select a parent segment.
  3. Open the target segment, right-mouse click, and then select Create Activation.
  4. In the Details panel, enter an Activation name and configure the activation according to the previous section on Configuration Parameters.
  5. Customize the activation output in the Output Mapping panel.

  • Attribute Columns
    • Select Export All Columns to export all columns without making any changes.
    • Select + Add Columns to add specific columns for the export. The Output Column Name pre-populates with the same Source column name. You can update the Output Column Name. Continue to select + Add Columnsto add new columns for your activation output.
  • String Builder
    • + Add string to create strings for export. Select from the following values:
      • String: Choose any value; use text to create a custom value.
      • Timestamp: The date and time of the export.
      • Segment Id: The segment ID number.
      • Segment Name: The segment name.
      • Audience Id: The parent segment number.
  1. Set a Schedule.

  • Select the values to define your schedule and optionally include email notifications.
  1. Select Create.

If you need to create an activation for a batch journey, review Creating a Batch Journey Activation.

Previous page
Next page