Data Connector for Facebook Ads reporting

The Data Connector for Facebook Ads reporting enables import of data from your Facebook Advertising campaign results.

Table of Contents

Prerequisites

  • Basic knowledge of Treasure Data
  • Basic knowledge of Facebook Marketing API, in particular Ads Insights API

Option 1: Use Web Console

Create a new connection

Please visit Treasure Data Connections and search and select Facebook. The dialog below will open.



We support two Authentication Methods:

Method 1: OAuth

Please select an existing OAuth connection for Facebook, or click the link under OAuth connection to create a new one.



Create a new OAuth connection

Please login to your Facebook account in popup window:



And grant access to Treasure Data app.

You will be redirected back to Treasure Data Connections. Please repeat the first step (Create a new connection) and choose your new OAuth connection.



Method 2: Access Token — deprecated, replaced by OAuth

Follow Step 1: Obtain Facebook Token under Option 2 to manually create access token.

Create a new transfer

Upon creating above connection, you will be automatically taken to My Connections tab. Look for the connection you created and click New Transfer.

The dialog below will open.



Fill in the following details and click on “Next”.

Ad Account ID

Ad Account ID from https://www.facebook.com/ads/manager, this is previously configurable as part of connection.

Data Level

Ads object level, available options are Campaign, Ad Set and Ad.

Metadata Fields

Metadata fields associated with the Ads object, avaiable fields are: Campaign Ad Set Ad

Default fields: created_time, effective_status, status



You can also access Ads object’s edges by using edge{field1, field2…}. For example to access fields from ad’s adcreatives edit: adcreatives{object_story_spec, url_tags, object_story_id, object_type, object_url}.

Insights Fields

Insight fields, available fields are: Insights



date_start and date_stop for Insights are always included. Insight values may be empty depending on the state of Ads object.

Breakdowns

Breakdown the insights response into different cohorts. Supported breakdowns are: age gender age, gender country region hourly_stats_aggregated_by_advertiser_time_zone



Filtering

Filters on the reported insights data. Format: Field —> Operator —> Value action_type supports IN operator with comma separate values, for example: post_like, like



Action Attribution Windows

Determines what is the attribution window for the actions. Supported values are: 1d_view, 7d_view, 28d_view, 1d_click, 7d_click, 28d_click, default([“1d_view”,“28d_click”]).



Incremental and fetch data since



By enabling incremental, you can define the duration (in days) of each incremental load. Default is 1 day. If fetch data since is not specified, default is set to past 30 days. Please note: data fetched will include data available up until midnight of the previous night.

Time Increment

If it is an integer, it is the number of days from 1 to 90. After you pick a reporting period by using “Fetch data since” and “Fetch data until”, you may choose to have the results for the whole period, or have results for smaller time slices. If “all_days” is used, it means one result set for the whole period. If “monthly” is used, you will get one result set for each calendar month in the given period. Or you can have one result set for each N-day period specified by this param.



API Version

The API version to match with your app if you are using your own app to generate access token.

Next, you will see a Preview of your data similar to the dialog below. Click on Next.



Third step is to select the database and table where you want to transfer the data, as per the following dialog:



Finally, specify the schedule of the data transfer using the dialog below and click Start Transfer:



You will see the new data transfer in progress listed under the My Input Transfers tab and a corresponding job will be listed in the Jobs section.

Now, you are ready to start analyzing your data!

Option 2: Use Command Line

Step 0: Install ‘td’ command

Install the newest Treasure Data Toolbelt.

Step 1: Obtain Facebook Token

Facebook provides 3 types of token. We recommend to use the never-expiring Page Access Token. You could also use User Token but it expires after 2 months even if it is extended. It is not batch-friendly.

How to get the never-expiring Page Access Token is described in here: https://www.rocketmarketinginc.com/blog/get-never-expiring-facebook-page-access-token/

Step 2: Prepare Configuration and Preview Data

First, prepare config.yml as below.

in:
  type: "facebook_ads_reporting"
  ad_account_id: '[your ad account id]'
  access_token: "[your Facebook access token]"
  api_version: "v2.8" #optional parameter, defaults v2.8
out:
  mode: append

For more details on available out modes, see Appendix.

Then, you can see a preview of the data using the preview command.

$ td connector:preview config.yml
+-----------------+---------------------+---------------+----------------+------------------+----------------------+---------------+------------------------------------+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+---------------------------+-------------------------+------------------+-------------------------+-------------+--------------------+-------------------------+--------------------------------+--------------------+---------------------------+--------------+----------------+----------------------+-----------------------+---------------------+----------------------------------------+-------------------------+---------------------------------+-----------------------------+--------------------------------+----------------------------------------+----------------------------+-----------------------------------+------------+-------------------+------------------+-----------------+-----------------------------+------------+-----------------+----------+------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------+-------------------------------+-------------------------------------------+-------------------------------------------+--------------------------------------+---------------------------+------------------------------+---------------------------+---------------------------+--------------------------------------------------------+---------------------------+
| account_id:long | account_name:string | ad_id:long    | ad_name:string | campaign_id:long | campaign_name:string | adset_id:long | adset_name:string                  | objective:string | actions:json                                                                                                                                                                                         | total_actions:long | total_unique_actions:long | total_action_value:long | impressions:long | social_impressions:long | clicks:long | social_clicks:long | unique_impressions:long | unique_social_impressions:long | unique_clicks:long | unique_social_clicks:long | spend:double | frequency:long | deeplink_clicks:long | app_store_clicks:long | website_clicks:long | cost_per_inline_post_engagement:double | inline_link_clicks:long | cost_per_inline_link_click:long | inline_post_engagement:long | unique_inline_link_clicks:long | cost_per_unique_inline_link_click:long | inline_link_click_ctr:long | unique_inline_link_click_ctr:long | reach:long | social_reach:long | ctr:double       | unique_ctr:long | unique_link_clicks_ctr:long | cpc:double | cpm:double      | cpp:long | cost_per_total_action:string | cost_per_action_type:json                                                                                                                                                                                                                            | cost_per_unique_click:long | estimated_ad_recall_rate:long | estimated_ad_recall_rate_lower_bound:long | estimated_ad_recall_rate_upper_bound:long | cost_per_estimated_ad_recallers:long | canvas_avg_view_time:long | canvas_avg_view_percent:long | date_start:timestamp      | date_stop:timestamp       | hourly_stats_aggregated_by_advertiser_time_zone:string | time:timestamp            |
+-----------------+---------------------+---------------+----------------+------------------+----------------------+---------------+------------------------------------+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+---------------------------+-------------------------+------------------+-------------------------+-------------+--------------------+-------------------------+--------------------------------+--------------------+---------------------------+--------------+----------------+----------------------+-----------------------+---------------------+----------------------------------------+-------------------------+---------------------------------+-----------------------------+--------------------------------+----------------------------------------+----------------------------+-----------------------------------+------------+-------------------+------------------+-----------------+-----------------------------+------------+-----------------+----------+------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------+-------------------------------+-------------------------------------------+-------------------------------------------+--------------------------------------+---------------------------+------------------------------+---------------------------+---------------------------+--------------------------------------------------------+---------------------------+
| 2xxxxxxxx       | "Foo Bar"           | 6xxxxxxxxxxxx | "FooBar"       | 6xxxxxxxxxxxx    | "Campaign"           | 6xxxxxxxxxxxx | "Campaign  (Web Remarketing)"      | "VIDEO_VIEWS"    | "[{\"action_type\":\"video_view\",\"value\":5},{\"action_type\":\"page_engagement\",\"value\":5},{\"action_type\":\"post_engagement\",\"value\":5}]"                                                 | 5                  | 0                         | 0                       | 25               | 0                       | 0           | 0                  | 0                       | 0                              | 0                  | 0                         | 0.23         | 0              | 0                    | 0                     | 0                   | 0.0                                    | 0                       | 0                               | 0                           | 0                              | 0                                      | 0                          | 0                                 | 0          | 0                 | 0.0              | 0               | 0                           | 0.0        | 9.2             | 0        | "0.046"                      | "[{\"action_type\":\"video_view\",\"value\":0.046},{\"action_type\":\"page_engagement\",\"value\":0.046},{\"action_type\":\"post_engagement\",\"value\":0.046}]"                                                                                     | 0                          | 0                             | 0                                         | 0                                         | 0                                    | 0                         | 0                            | "2016-07-28 07:00:00 UTC" | "2016-07-28 07:00:00 UTC" | "19:00:00 - 19:59:59"                                  | "2016-07-29 02:00:00 UTC" |
| 2xxxxxxxx       | "Foo Bar"           | 6xxxxxxxxxxxx | "FooBar"       | 6xxxxxxxxxxxx    | "Campaign"           | 6xxxxxxxxxxxx | "Campaign  (Web Remarketing)"      | "VIDEO_VIEWS"    | "[{\"action_type\":\"video_play\",\"value\":1},{\"action_type\":\"video_view\",\"value\":15},{\"action_type\":\"page_engagement\",\"value\":15},{\"action_type\":\"post_engagement\",\"value\":15}]" | 15                 | 0                         | 0                       | 114              | 1                       | 1           | 0                  | 0                       | 0                              | 0                  | 0                         | 0.32         | 0              | 0                    | 0                     | 0                   | 0.0                                    | 0                       | 0                               | 0                           | 0                              | 0                                      | 0                          | 0                                 | 0          | 0                 | 0.87719298245614 | 0               | 0                           | 0.32       | 2.8070175438596 | 0        | "0.021333333333333"          | "[{\"action_type\":\"video_play\",\"value\":0.32},{\"action_type\":\"video_view\",\"value\":0.021333333333333},{\"action_type\":\"page_engagement\",\"value\":0.021333333333333},{\"action_type\":\"post_engagement\",\"value\":0.021333333333333}]" | 0                          | 0                             | 0                                         | 0                                         | 0                                    | 0                         | 0                            | "2016-07-28 07:00:00 UTC" | "2016-07-28 07:00:00 UTC" | "20:00:00 - 20:59:59"                                  | "2016-07-29 03:00:00 UTC" |
| 2xxxxxxxx       | "Foo Bar"           | 6xxxxxxxxxxxx | "FooBar"       | 6xxxxxxxxxxxx    | "Campaign"           | 6xxxxxxxxxxxx | "Campaign  (Web Remarketing)"      | "VIDEO_VIEWS"    | "[{\"action_type\":\"video_play\",\"value\":1},{\"action_type\":\"video_view\",\"value\":87},{\"action_type\":\"page_engagement\",\"value\":87},{\"action_type\":\"post_engagement\",\"value\":87}]" | 87                 | 0                         | 0                       | 452              | 9                       | 4           | 0                  | 0                       | 0                              | 0                  | 0                         | 1.95         | 0              | 0                    | 0                     | 0                   | 0.4875                                 | 0                       | 0                               | 4                           | 0                              | 0                                      | 0                          | 0                                 | 0          | 0                 | 0.88495575221239 | 0               | 0                           | 0.4875     | 4.3141592920354 | 0        | "0.022413793103448"          | "[{\"action_type\":\"video_play\",\"value\":1.95},{\"action_type\":\"video_view\",\"value\":0.022413793103448},{\"action_type\":\"page_engagement\",\"value\":0.022413793103448},{\"action_type\":\"post_engagement\",\"value\":0.022413793103448}]" | 0                          | 0                             | 0                                         | 0                                         | 0                                    | 0                         | 0                            | "2016-07-28 07:00:00 UTC" | "2016-07-28 07:00:00 UTC" | "21:00:00 - 21:59:59"                                  | "2016-07-29 04:00:00 UTC" |
| 2xxxxxxxx       | "Foo Bar"           | 6xxxxxxxxxxxx | "FooBar"       | 6xxxxxxxxxxxx    | "Campaign"           | 6xxxxxxxxxxxx | "Campaign  (Web Remarketing)"      | "VIDEO_VIEWS"    | "[{\"action_type\":\"video_view\",\"value\":3},{\"action_type\":\"page_engagement\",\"value\":3},{\"action_type\":\"post_engagement\",\"value\":3}]"                                                 | 3                  | 0                         | 0                       | 9                | 1                       | 0           | 0                  | 0                       | 0                              | 0                  | 0                         | 0.01         | 0              | 0                    | 0                     | 0                   | 0.0                                    | 0                       | 0                               | 0                           | 0                              | 0                                      | 0                          | 0                                 | 0          | 0                 | 0.0              | 0               | 0                           | 0.0        | 1.1111111111111 | 0        | "0.0033333333333333"         | "[{\"action_type\":\"video_view\",\"value\":0.0033333333333333},{\"action_type\":\"page_engagement\",\"value\":0.0033333333333333},{\"action_type\":\"post_engagement\",\"value\":0.0033333333333333}]"                                              | 0                          | 0                             | 0                                         | 0                                         | 0                                    | 0                         | 0                            | "2016-07-28 07:00:00 UTC" | "2016-07-28 07:00:00 UTC" | "22:00:00 - 22:59:59"                                  | "2016-07-29 05:00:00 UTC" |
| 2xxxxxxxx       | "Foo Bar"           | 6xxxxxxxxxxxx | "FooBar"       | 6xxxxxxxxxxxx    | "Campaign"           | 6xxxxxxxxxxxx | "Campaign  (Web Remarketing)"      | "VIDEO_VIEWS"    | "[{\"action_type\":\"video_view\",\"value\":40},{\"action_type\":\"page_engagement\",\"value\":40},{\"action_type\":\"post_engagement\",\"value\":40}]"                                              | 40                 | 0                         | 0                       | 230              | 8                       | 0           | 0                  | 0                       | 0                              | 0                  | 0                         | 1.16         | 0              | 0                    | 0                     | 0                   | 0.0                                    | 0                       | 0                               | 0                           | 0                              | 0                                      | 0                          | 0                                 | 0          | 0                 | 0.0              | 0               | 0                           | 0.0        | 5.0434782608696 | 0        | "0.029"                      | "[{\"action_type\":\"video_view\",\"value\":0.029},{\"action_type\":\"page_engagement\",\"value\":0.029},{\"action_type\":\"post_engagement\",\"value\":0.029}]"                                                                                     | 0                          | 0                             | 0                                         | 0                                         | 0                                    | 0                         | 0                            | "2016-07-28 07:00:00 UTC" | "2016-07-28 07:00:00 UTC" | "23:00:00 - 23:59:59"                                  | "2016-07-29 06:00:00 UTC" |
| 2xxxxxxxx       | "Foo Bar"           | 6xxxxxxxxxxxx | "FooBar"       | 6xxxxxxxxxxxx    | "Campaign"           | 6xxxxxxxxxxxx | "Campaign  (Web Remarketing)"      | "VIDEO_VIEWS"    | "[{\"action_type\":\"video_view\",\"value\":1},{\"action_type\":\"page_engagement\",\"value\":1},{\"action_type\":\"post_engagement\",\"value\":1}]"                                                 | 1                  | 0                         | 0                       | 17               | 0                       | 0           | 0                  | 0                       | 0                              | 0                  | 0                         | 0.06         | 0              | 0                    | 0                     | 0                   | 0.0                                    | 0                       | 0                               | 0                           | 0                              | 0                                      | 0                          | 0                                 | 0          | 0                 | 0.0              | 0               | 0                           | 0.0        | 3.5294117647059 | 0        | "0.06"                       | "[{\"action_type\":\"video_view\",\"value\":0.06},{\"action_type\":\"page_engagement\",\"value\":0.06},{\"action_type\":\"post_engagement\",\"value\":0.06}]"                                                                                        | 0                          | 0                             | 0                                         | 0                                         | 0                                    | 0                         | 0                            | "2016-07-29 07:00:00 UTC" | "2016-07-29 07:00:00 UTC" | "00:00:00 - 00:59:59"                                  | "2016-07-29 07:00:00 UTC" |
| 2xxxxxxxx       | "Foo Bar"           | 6xxxxxxxxxxxx | "FooBar"       | 6xxxxxxxxxxxx    | "Campaign"           | 6xxxxxxxxxxxx | "Campaign  (Web Remarketing)"      | "VIDEO_VIEWS"    | "[{\"action_type\":\"video_view\",\"value\":14},{\"action_type\":\"page_engagement\",\"value\":14},{\"action_type\":\"post_engagement\",\"value\":14}]"                                              | 14                 | 0                         | 0                       | 72               | 2                       | 0           | 0                  | 0                       | 0                              | 0                  | 0                         | 0.36         | 0              | 0                    | 0                     | 0                   | 0.0                                    | 0                       | 0                               | 0                           | 0                              | 0                                      | 0                          | 0                                 | 0          | 0                 | 0.0              | 0               | 0                           | 0.0        | 5.0             | 0        | "0.025714285714286"          | "[{\"action_type\":\"video_view\",\"value\":0.025714285714286},{\"action_type\":\"page_engagement\",\"value\":0.025714285714286},{\"action_type\":\"post_engagement\",\"value\":0.025714285714286}]"                                                 | 0                          | 0                             | 0                                         | 0                                         | 0                                    | 0                         | 0                            | "2016-07-29 07:00:00 UTC" | "2016-07-29 07:00:00 UTC" | "01:00:00 - 01:59:59"                                  | "2016-07-29 08:00:00 UTC" |
| 2xxxxxxxx       | "Foo Bar"           | 6xxxxxxxxxxxx | "FooBar"       | 6xxxxxxxxxxxx    | "Campaign"           | 6xxxxxxxxxxxx | "Campaign  (Web Remarketing)"      | "VIDEO_VIEWS"    | "[{\"action_type\":\"video_view\",\"value\":23},{\"action_type\":\"page_engagement\",\"value\":23},{\"action_type\":\"post_engagement\",\"value\":23}]"                                              | 23                 | 0                         | 0                       | 96               | 4                       | 0           | 0                  | 0                       | 0                              | 0                  | 0                         | 0.47         | 0              | 0                    | 0                     | 0                   | 0.0                                    | 0                       | 0                               | 0                           | 0                              | 0                                      | 0                          | 0                                 | 0          | 0                 | 0.0              | 0               | 0                           | 0.0        | 4.8958333333333 | 0        | "0.020434782608696"          | "[{\"action_type\":\"video_view\",\"value\":0.020434782608696},{\"action_type\":\"page_engagement\",\"value\":0.020434782608696},{\"action_type\":\"post_engagement\",\"value\":0.020434782608696}]"                                                 | 0                          | 0                             | 0                                         | 0                                         | 0                                    | 0                         | 0                            | "2016-07-29 07:00:00 UTC" | "2016-07-29 07:00:00 UTC" | "10:00:00 - 10:59:59"                                  | "2016-07-29 17:00:00 UTC" |
| 2xxxxxxxx       | "Foo Bar"           | 6xxxxxxxxxxxx | "FooBar"       | 6xxxxxxxxxxxx    | "Campaign"           | 6xxxxxxxxxxxx | "Campaign  (Web Remarketing)"      | "VIDEO_VIEWS"    | "[{\"action_type\":\"video_view\",\"value\":33},{\"action_type\":\"page_engagement\",\"value\":33},{\"action_type\":\"post_engagement\",\"value\":33}]"                                              | 33                 | 0                         | 0                       | 112              | 1                       | 0           | 0                  | 0                       | 0                              | 0                  | 0                         | 0.64         | 0              | 0                    | 0                     | 0                   | 0.0                                    | 0                       | 0                               | 0                           | 0                              | 0                                      | 0                          | 0                                 | 0          | 0                 | 0.0              | 0               | 0                           | 0.0        | 5.7142857142857 | 0        | "0.019393939393939"          | "[{\"action_type\":\"video_view\",\"value\":0.019393939393939},{\"action_type\":\"page_engagement\",\"value\":0.019393939393939},{\"action_type\":\"post_engagement\",\"value\":0.019393939393939}]"                                                 | 0                          | 0                             | 0                                         | 0                                         | 0                                    | 0                         | 0                            | "2016-07-29 07:00:00 UTC" | "2016-07-29 07:00:00 UTC" | "11:00:00 - 11:59:59"                                  | "2016-07-29 18:00:00 UTC" |
| 2xxxxxxxx       | "Foo Bar"           | 6xxxxxxxxxxxx | "FooBar"       | 6xxxxxxxxxxxx    | "Campaign"           | 6xxxxxxxxxxxx | "Campaign  (Web Remarketing)"      | "VIDEO_VIEWS"    | "[{\"action_type\":\"video_view\",\"value\":23},{\"action_type\":\"page_engagement\",\"value\":23},{\"action_type\":\"post_engagement\",\"value\":23}]"                                              | 23                 | 0                         | 0                       | 83               | 2                       | 1           | 0                  | 0                       | 0                              | 0                  | 0                         | 0.43         | 0              | 0                    | 0                     | 0                   | 0.43                                   | 0                       | 0                               | 1                           | 0                              | 0                                      | 0                          | 0                                 | 0          | 0                 | 1.2048192771084  | 0               | 0                           | 0.43       | 5.1807228915663 | 0        | "0.018695652173913"          | "[{\"action_type\":\"video_view\",\"value\":0.018695652173913},{\"action_type\":\"page_engagement\",\"value\":0.018695652173913},{\"action_type\":\"post_engagement\",\"value\":0.018695652173913}]"                                                 | 0                          | 0                             | 0                                         | 0                                         | 0                                    | 0                         | 0                            | "2016-07-29 07:00:00 UTC" | "2016-07-29 07:00:00 UTC" | "12:00:00 - 12:59:59"                                  | "2016-07-29 19:00:00 UTC" |
| 2xxxxxxxx       | "Foo Bar"           | 6xxxxxxxxxxxx | "FooBar"       | 6xxxxxxxxxxxx    | "Campaign"           | 6xxxxxxxxxxxx | "Campaign  (Web Remarketing)"      | "VIDEO_VIEWS"    | "[{\"action_type\":\"video_view\",\"value\":49},{\"action_type\":\"page_engagement\",\"value\":49},{\"action_type\":\"post_engagement\",\"value\":49}]"                                              | 49                 | 0                         | 0                       | 199              | 2                       | 1           | 0                  | 0                       | 0                              | 0                  | 0                         | 1.23         | 0              | 0                    | 0                     | 0                   | 0.0                                    | 0                       | 0                               | 0                           | 0                              | 0                                      | 0                          | 0                                 | 0          | 0                 | 0.50251256281407 | 0               | 0                           | 1.23       | 6.1809045226131 | 0        | "0.025102040816327"          | "[{\"action_type\":\"video_view\",\"value\":0.025102040816327},{\"action_type\":\"page_engagement\",\"value\":0.025102040816327},{\"action_type\":\"post_engagement\",\"value\":0.025102040816327}]"                                                 | 0                          | 0                             | 0                                         | 0                                         | 0                                    | 0                         | 0                            | "2016-07-29 07:00:00 UTC" | "2016-07-29 07:00:00 UTC" | "13:00:00 - 13:59:59"                                  | "2016-07-29 20:00:00 UTC" |
| 2xxxxxxxx       | "Foo Bar"           | 6xxxxxxxxxxxx | "FooBar"       | 6xxxxxxxxxxxx    | "Campaign"           | 6xxxxxxxxxxxx | "Campaign  (Web Remarketing)"      | "VIDEO_VIEWS"    | "[{\"action_type\":\"video_view\",\"value\":22},{\"action_type\":\"page_engagement\",\"value\":22},{\"action_type\":\"post_engagement\",\"value\":22}]"                                              | 22                 | 0                         | 0                       | 119              | 3                       | 0           | 0                  | 0                       | 0                              | 0                  | 0                         | 0.66         | 0              | 0                    | 0                     | 0                   | 0.0                                    | 0                       | 0                               | 0                           | 0                              | 0                                      | 0                          | 0                                 | 0          | 0                 | 0.0              | 0               | 0                           | 0.0        | 5.546218487395  | 0        | "0.03"                       | "[{\"action_type\":\"video_view\",\"value\":0.03},{\"action_type\":\"page_engagement\",\"value\":0.03},{\"action_type\":\"post_engagement\",\"value\":0.03}]"                                                                                        | 0                          | 0                             | 0                                         | 0                                         | 0                                    | 0                         | 0                            | "2016-07-29 07:00:00 UTC" | "2016-07-29 07:00:00 UTC" | "14:00:00 - 14:59:59"                                  | "2016-07-29 21:00:00 UTC" |
| 2xxxxxxxx       | "Foo Bar"           | 6xxxxxxxxxxxx | "FooBar"       | 6xxxxxxxxxxxx    | "Campaign"           | 6xxxxxxxxxxxx | "Campaign  (Web Remarketing)"      | "VIDEO_VIEWS"    | "[{\"action_type\":\"video_view\",\"value\":27},{\"action_type\":\"page_engagement\",\"value\":27},{\"action_type\":\"post_engagement\",\"value\":27}]"                                              | 27                 | 0                         | 0                       | 97               | 2                       | 2           | 0                  | 0                       | 0                              | 0                  | 0                         | 0.81         | 0              | 0                    | 0                     | 0                   | 0.81                                   | 0                       | 0                               | 1                           | 0                              | 0                                      | 0                          | 0                                 | 0          | 0                 | 2.0618556701031  | 0               | 0                           | 0.405      | 8.3505154639175 | 0        | "0.03"                       | "[{\"action_type\":\"video_view\",\"value\":0.03},{\"action_type\":\"page_engagement\",\"value\":0.03},{\"action_type\":\"post_engagement\",\"value\":0.03}]"                                                                                        | 0                          | 0                             | 0                                         | 0                                         | 0                                    | 0                         | 0                            | "2016-07-29 07:00:00 UTC" | "2016-07-29 07:00:00 UTC" | "15:00:00 - 15:59:59"                                  | "2016-07-29 22:00:00 UTC" |
| 2xxxxxxxx       | "Foo Bar"           | 6xxxxxxxxxxxx | "FooBar"       | 6xxxxxxxxxxxx    | "Campaign"           | 6xxxxxxxxxxxx | "Campaign  (Web Remarketing)"      | "VIDEO_VIEWS"    | "[{\"action_type\":\"video_view\",\"value\":40},{\"action_type\":\"page_engagement\",\"value\":40},{\"action_type\":\"post_engagement\",\"value\":40}]"                                              | 40                 | 0                         | 0                       | 172              | 4                       | 1           | 0                  | 0                       | 0                              | 0                  | 0                         | 1.18         | 0              | 0                    | 0                     | 0                   | 1.18                                   | 0                       | 0                               | 1                           | 0                              | 0                                      | 0                          | 0                                 | 0          | 0                 | 0.58139534883721 | 0               | 0                           | 1.18       | 6.8604651162791 | 0        | "0.0295"                     | "[{\"action_type\":\"video_view\",\"value\":0.0295},{\"action_type\":\"page_engagement\",\"value\":0.0295},{\"action_type\":\"post_engagement\",\"value\":0.0295}]"                                                                                  | 0                          | 0                             | 0                                         | 0                                         | 0                                    | 0                         | 0                            | "2016-07-29 07:00:00 UTC" | "2016-07-29 07:00:00 UTC" | "16:00:00 - 16:59:59"                                  | "2016-07-29 23:00:00 UTC" |
| 2xxxxxxxx       | "Foo Bar"           | 6xxxxxxxxxxxx | "FooBar"       | 6xxxxxxxxxxxx    | "Campaign"           | 6xxxxxxxxxxxx | "Campaign  (Web Remarketing)"      | "VIDEO_VIEWS"    | "[{\"action_type\":\"video_view\",\"value\":20},{\"action_type\":\"page_engagement\",\"value\":20},{\"action_type\":\"post_engagement\",\"value\":20}]"                                              | 20                 | 0                         | 0                       | 126              | 0                       | 1           | 0                  | 0                       | 0                              | 0                  | 0                         | 2.35         | 0              | 0                    | 0                     | 0                   | 2.35                                   | 0                       | 0                               | 1                           | 0                              | 0                                      | 0                          | 0                                 | 0          | 0                 | 0.79365079365079 | 0               | 0                           | 2.35       | 18.650793650794 | 0        | "0.1175"                     | "[{\"action_type\":\"video_view\",\"value\":0.1175},{\"action_type\":\"page_engagement\",\"value\":0.1175},{\"action_type\":\"post_engagement\",\"value\":0.1175}]"                                                                                  | 0                          | 0                             | 0                                         | 0                                         | 0                                    | 0                         | 0                            | "2016-07-29 07:00:00 UTC" | "2016-07-29 07:00:00 UTC" | "17:00:00 - 17:59:59"                                  | "2016-07-30 00:00:00 UTC" |
+-----------------+---------------------+---------------+----------------+------------------+----------------------+---------------+------------------------------------+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+---------------------------+-------------------------+------------------+-------------------------+-------------+--------------------+-------------------------+--------------------------------+--------------------+---------------------------+--------------+----------------+----------------------+-----------------------+---------------------+----------------------------------------+-------------------------+---------------------------------+-----------------------------+--------------------------------+----------------------------------------+----------------------------+-----------------------------------+------------+-------------------+------------------+-----------------+-----------------------------+------------+-----------------+----------+------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------+-------------------------------+-------------------------------------------+-------------------------------------------+--------------------------------------+---------------------------+------------------------------+---------------------------+---------------------------+--------------------------------------------------------+---------------------------+
15 rows in set
Untitled-3
We don't need to run td connector:guess for the facebook_ads_reporting connector.

Step 3: Execute Load Job

Finally, submit the load job. It may take a couple of hours depending on the size of the data.

Facebook connector provides time column automatically. You don’t have to specify the --time-column option.

$ td connector:issue load.yml --database td_sample_db --table td_sample_table

Scheduled Execution

You can schedule periodic Data Connector execution for incremental Facebook Ads reporting data. We take great care in distributing and operating our scheduler in order to achieve high availability. By using this feature, you no longer need a cron daemon on your local datacenter.

For the scheduled import at first run, the Data Connector for Facebook Ads reporting imports all of your ad data.

On the second and subsequent runs, it will only import data that is newer than the last load.

Create the schedule

A new schedule can be created using the td connector:create command. The following are required: the name of the schedule, the cron-style schedule, the database and table where their data will be stored, and the Data Connector configuration file.

$ td connector:create \
    daily_import \
    "10 0 * * *" \
    td_sample_db \
    td_sample_table \
    config.yml
Untitled-3
The `cron` parameter also accepts three special options: `@hourly`, `@daily` and `@monthly`.
Untitled-3
By default, schedule is setup in UTC timezone. You can set the schedule in a timezone using -t or --timezone option. Please note that `--timezone` option only supports extended timezone formats like 'Asia/Tokyo', 'America/Los_Angeles' etc. Timezone abbreviations like PST, CST are *not* supported and may lead to unexpected schedules.

List All Schedules

You can see the list of all current schedule entries with the command td connector:list.

$ td connector:list
+--------------+------------+----------+-------+--------------+-----------------+----------------------------+
| Name         | Cron       | Timezone | Delay | Database     | Table           | Config                     |
+--------------+------------+----------+-------+--------------+-----------------+----------------------------+
| daily_import | 10 0 * * * | UTC      | 0     | td_sample_db | td_sample_table | {"in"=>{"type"=>"fac", ... |
+--------------+------------+----------+-------+--------------+-----------------+----------------------------+

Show Schedule Settings And History

td connector:show shows the execution settings of a schedule entry.

% td connector:show daily_import
Name     : daily_import
Cron     : 10 0 * * *
Timezone : UTC
Delay    : 0
Database : td_sample_db
Table    : td_sample_table
Config
---
in:
  type: "facebook_ads_reporting"
  ad_account_id: '[your ad account id]'
  access_token: "[your Facebook access token]"
  api_version: "v2.8" #optional parameter, defaults v2.8
out:
  mode: append

td connector:history shows the execution history of a schedule entry. To investigate the results of each individual run, please use td job <jobid>.

% td connector:history daily_import
+--------+---------+---------+--------------+-----------------+----------+---------------------------+----------+
| JobID  | Status  | Records | Database     | Table           | Priority | Started                   | Duration |
+--------+---------+---------+--------------+-----------------+----------+---------------------------+----------+
| 578066 | success | 10000   | td_sample_db | td_sample_table | 0        | 2015-04-18 00:10:05 +0000 | 160      |
| 577968 | success | 10000   | td_sample_db | td_sample_table | 0        | 2015-04-17 00:10:07 +0000 | 161      |
| 577914 | success | 10000   | td_sample_db | td_sample_table | 0        | 2015-04-16 00:10:03 +0000 | 152      |
| 577872 | success | 10000   | td_sample_db | td_sample_table | 0        | 2015-04-15 00:10:04 +0000 | 163      |
| 577810 | success | 10000   | td_sample_db | td_sample_table | 0        | 2015-04-14 00:10:04 +0000 | 164      |
| 577766 | success | 10000   | td_sample_db | td_sample_table | 0        | 2015-04-13 00:10:04 +0000 | 155      |
| 577710 | success | 10000   | td_sample_db | td_sample_table | 0        | 2015-04-12 00:10:05 +0000 | 156      |
| 577610 | success | 10000   | td_sample_db | td_sample_table | 0        | 2015-04-11 00:10:04 +0000 | 157      |
+--------+---------+---------+--------------+-----------------+----------+---------------------------+----------+
8 rows in set

Delete Schedule

td connector:delete will remove the schedule.

$ td connector:delete daily_import

Configuration

See the following table for more details on available in modes.

Option name Description Type Required? Default value
ad_account_id Ad account ID of access_token string yes
access_token Access Token. We recommend to use won’t expired token (see below section) string yes
app_secret If set, appsecret_proof parameter will be used to access API string optional
metadata_fields What you want to obtain metadata fields for given object. Available fields are here array optional created_time, status, effective_status
fields What you want to obtain fields for insights. Available fields are here array optional
breakdowns see breakdowns for details string optional
filtering see filter for details array optional
action_attribution_windows Determines what is the attribution window for the actions. For example, 28d_click means the API returns all actions that happened 28 days after someone clicked on the ad. array optional
retry_limit Trying maximum retry times integer optional 5
retry_initial_wait_sec Wait seconds for exponential backoff initial value integer optional 10
last_fetched_at Only imports data since this date, it will be 30 days from today if not specified, ie. (today - 30 days). Recommended formats: %m/%d/%Y or %Y-%m-%dT%H:%M:%S.%L%Z string optional
incremental true for generate “config_diff” with embulk run -c config.diff bool optional true
duration # of days from start date to pull in insights data. See Appendix B for more details. integer optional 1
time_increment If it is an integer, it is the number of days from 1 to 90. After you pick a reporting period by using last_fetched_at and last_fetched_until, you may choose to have the results for the whole period, or have results for smaller time slices. If “all_days” is used, it means one result set for the whole period. If “monthly” is used, you will get one result set for each calendar month in the given period. Or you can have one result set for each N-day period specified by this param. See Appendix B for more details. string optional 1
last_fetched_until Only imports data until this date, today if not specified. Recommended formats: see last_fetched_at. See Appendix C for more details. string optional
add_time_column If true, “time” column will be added automatically, based on date_stop value of insights, otherwise, “time” column will be added by upload_time bool optional true
data_level Obtain data level. Facebook insights level: Campaign –> Adset –> Ad. Available options are “campaign”, “adset” or “ad”. string optional “ad”
api_version Use Facebook API version. Default is v2.8. If specified version is older than v2.8 (deprecated), plugin will use “v2.8” string optional “v2.8”
limit_per_page Number of records per page, reduce this value when you encouter error message “Please reduce the amount of data you’re asking for, then retry your request” integer optional 100000

Appendix

A) Modes for out plugin

You can specify file import mode in out section of seed.yml.

append (default)

This is the default mode and records are appended to the target table.

in:
  ...
out:
  mode: append

replace (In td 0.11.10 and later)

This mode replaces data in the target table. Please note that any manual schema changes made to the target table will remain intact with this mode.

in:
  ...
out:
  mode: replace

B) duration and time_increment

When to use duration?
  • Firstly, duration only has effect when incremental = true
  • duration will be used to compute time range for your request, based on last_fetched_at, for example, if last_fetched_at = 02/01/2017 and duration = 2, you’re requesting for a time range of: 02/01/2017 → 02/02/2017

Directly translated to Facebook request parameters:

.time_range({"since":"2017-02-01","until":"2017-02-02"})
  • There is no breakdown behavior when you use this parameter. It will return insight values for the whole time range.
  • Note: Keep in mind, when no last_fetched_at is specified, it will be 30 days from today, ie. (today – 30 days)
When to use time_increment?
  • time_increment is used when you want to breakdown insights data by a number of days. Its effect will be most easy to recognize without incremental mode and duration.

Example:

incremental: false
last_fetched_at: 03/01/2017
last_fetched_until: 03/31/2017
time_increment: 1

Using above configurations, you can import a whole March 2017 by one run, and still have it breakdown daily.

duration and time_increment differences
  • duration is used for incremental mode
  • time_increment is an official Facebook parameter
  • Important note: If both duration and time_increment are available (which is default), duration will have higher priority than time_increment when its value is less than time_increment. With that being said, you can’t have breakdown data per 2 days (time_increment = 2) when duration = 1, result will be 1-day data. In the other hand, you can perfectly breakdown data daily (time_increment = 1) for a 2-day duration (duration = 2).

C) When to use last_fetched_until?

  • last_fetched_until is optional, and its only usage is to limit imported data to an upper date.

Example: you have a job to import March 2017, and it’s scheduled to run repeatedly to get daily data, below is the configurations to achieve it:

incremental: true
duration: 1
last_fetched_at: 03/01/2017
last_fetched_until: 03/31/2017

Note: As you can see, you can simulate time_increment effect by using duration and run the job repeatedly, but it will cost you more requests (Request Rate Limit will occur faster).

  • If your scheduled job (eventually) exceeds configured last_fetched_until, an empty result will be returned.

Last modified: Mar 23 2017 05:49:25 UTC

If this article is incorrect or outdated, or omits critical information, please let us know. For all other issues, please see our support channels.