Skip to content
Last updated

Facebook Ads Insights Import Integration

The Data Connector for Facebook Ads Insights (previous known as Facebook Ads Reporting) enables the import of data from your Facebook Advertising campaign results.

Prerequisites

  • Basic knowledge of Treasure Data
  • Basic knowledge of Facebook Marketing API, in particular Ads Insights API.
  • Admin or Advertiser is required as admin permission

Use TD Console

Create a new connection

Go to Treasure Data Connections and search and select Facebook Ads. The following dialog opens.

We support two Authentication Methods:

  • OAuth
  • Access Token

Using OAuth

Select an existing OAuth connection for Facebook or select the link under OAuth connection to create a new one.

  • Create a new OAuth connection

Log into your Facebook account in a popup window:

And grant access to the Treasure Data app.

You will be redirected back to Treasure Data Connections. Repeat the first step (Create a new connection) and choose your new OAuth connection.

Using an Access Token

See Obtain Facebook Token under Section 3 to create an access token.

Create a new transfer

After creating the connection, you are automatically taken to the Authentications tab. Look for the connection you created and select New Source.

The following dialog opens.

Edit the following details:

  • Ad Account ID
  • Data Level
  • Import Metadata for Related Objects
  • Import all Insights or Insights Fields
  • Breakdowns
  • Filtering
  • Action Attribution Windows
  • Start and End Date or Incremental
  • Reporting Period

Details on these parameters are as follows:

Ad Account ID

Add your Ad Account ID, which you can find as described here.

Data Level

The Ads object level. Available options are Campaign, Ad Set, and Ad.

By default, Import metadata for related objects is unchecked and the connector imports only fundamental metadata fields: created_time, effective_status, status.

By checking it, you can specify related objects or individual metadata fields to import.

Select All Metadata Fields to import all related edges. You can select which edges to import by adding each edge in Fetch Metadata.

Metadata Fields

Metadata fields associated with the Ads objects. Selecting Individual Metadata Fields allows you to import only the fields you want. Select fields under Metadata Fields.

You can also access Ads object’s edges by using edge{field1, field2…}.

Insights Fields

Insight fields provide data about the performance of your campaign.

By default, Import all insights fields are checked.

You can uncheck Import all insights fields to select specific fields.

The Insights fields date_start and date_stop are added by default, if not explicitly set. Insight values might be empty depending on the state of Ads object. No errors are generated if the fields do not contain values.

Breakdowns

Breakdowns are used to capture Ads Insights responses that are grouped into different sets or cohorts. Supported breakdowns are:

  • age

  • gender

  • country

  • region

  • hourly_stats_aggregated_by_advertiser_time_zone

Filtering

Filters on the reported Insights data. Format: Field —> Operator —> Value, the action_type supports IN operator with comma separate values, i.e. post_like, like

Action Attribution Windows

Determines the attribution window for the actions. Supported values are:

  • 1d_view
  • 7d_view
  • 28d_view
  • 1d_click
  • 7d_click
  • 28d_click

Defaults values are: ([“1d_view”,“28d_click”]) if not explicitly set.

Incremental Import and Start Date

Specify if you want Incremental import and Start Date.

By enabling incremental, you can define the duration (in days) of each incremental load. The default is 1 day. If Start Date is not specified, then the default is set to past 30 days.

The data that is fetched includes data that is available up until midnight of the previous night.

Reporting Period

After you pick a reporting period by using Start Date and End Date, you may choose to get results for the whole period or get results for smaller time slices. If you select All Days, then you get one result set for the whole period. If you specify monthly, then you get one result set for each calendar month in the given period.

Or you can get one result set for each N-day period specified by selecting In Days. You can specify the number of days from 1 to 90.

After you complete specifying parameters, select Next. You then see a Preview of your data similar to the dialog below. Select Next. Select Preview while Importing Data if you cannot move beyond Preview.

Select the database and table where you want to transfer the data, as shown in the following dialog:

Specify the schedule of the data transfer using the dialog below and select Start Transfer.

You see the new data transfer in progress listed under the My Input Transfers tab and a corresponding job is listed in the Jobs section.

Now, you are ready to start analyzing your data!

Use Command Line

Install TD Toolbelt

Install the newest TD Toolbelt.

Obtain Facebook Token

Facebook provides 3 types of tokens. We recommend using the never-expiring Page Access Token. You can also use User Token but it expires after 2 months even if it is extended. It is not batch-friendly.

For steps that describe how to get the never-expiring Page Access Token, see Facebook Long-Lived Access Tokens.

Prepare Configuration and Preview Data

We don't need to run td connector:guess for the facebook_ads_reporting connector.

Prepare config.yml as follows:

in:
  type: "facebook_ads_reporting"
  ad_account_id: '[your ad account id]'
  access_token: "[your Facebook access token]"
  data_level: ad
  fields:
    - clicks
    - impressions
  metadata_import: all
  metadata_edge:
    - adrules_governed
    - copies
    - keywordstats
    - leads
    - targetingsentencelines
  filtering:
    - {field : "clicks", operator: "GREATER_THAN", value: 10}
    - {field : "impressions", operator: "GREATER_THAN", value: 200}
out:
  mode: append

You can see a preview of the data using the preview command.

td connector:preview config.yml

Configuration

See the following table for more details on available modes.

Option nameDescriptionTypeRequired?Default value
ad_account_idAd account ID of access_tokenstringyes
access_tokenAccess Token. We recommend using the non-expiring token (see the next section)stringyes
app_secretIf set, appsecret_proof parameter is used to access APIstringoptional
metadata_importImport related edges or individual metadata fields? Available options are all and individualstringoptionalindividual
metadata_edgeRelated edges to import. Only effective when metadata_import is all, see related edges.arrayoptional
metadata_fieldsWhat metadata fields you want to obtain for a given object. Review available fields.arrayoptionalcreated_time, status, effective_status
fieldsWhat fields you want to obtain for insights. Review available fields.arrayoptional
breakdownsSee breakdowns for details.stringoptional
filteringSee filter for details.arrayoptional
action_attribution_windowsDetermines the attribution window for the actions. For example, 28d_click Means the API returns all actions that happened 28 days after someone clicked on the ad.arrayoptional
retry_limitTrying maximum retry timesintegeroptional5
retry_initial_wait_secWait seconds for exponential backoff initial valueintegeroptional10
last_fetched_atOnly imports data since this date, it is 30 days from today if not specified, ie. (today - 30 days). Recommended formats: %m/%d/%Y or %Y-%m-%dT%H:%M:%S.%L%Zstringoptional
incrementaltrue for generate “config_diff” with embulk run -c config.diffbooloptionaltrue
duration# of days from start date to pull in insights data. See Appendix B for more details.integeroptional1
time_incrementIf it is an integer, it is the number of days from 1 to 90. After you pick a reporting period by using last_fetched_at and last_fetched_until, you might choose to have the results for the whole period, or have results for smaller time slices. If “all_days” is used, it means one result set for the whole period. If “monthly” is used, you get one result set for each calendar month in the given period. Or you can have one result set for each N-day period specified by this parameter. See Appendix B for more details.stringoptional1
last_fetched_untilOnly imports data until this date, today if not specified. Recommended formats: see last_fetched_at. See Appendix C for more details.stringoptional
add_time_columnIf true, “time” column is added automatically, based on date_stop value of insights, otherwise, “time” column is added by upload_timebooloptionaltrue
data_levelObtain data level. Facebook insights level: Campaign –> Adset –> Ad. Available options are “campaign”, “adset”, “ad”, or “ad_creative”.stringoptional“ad”
api_versionUse Facebook API version. Default is v3.2. If specified version is older than v3.1 (deprecated), plugin uses “v3.2”stringoptional“v3.2”
limit_per_pageNumber of records per page, reduce this value when you encounter error message “Please reduce the amount of data you’re asking for, then retry your request”integeroptional100000

Campaign

  • ad_studies
  • adrules_governed
  • ads
  • adsets
  • copies

AdSet

  • activities
  • ad_studies
  • adrules_governed
  • ads
  • copies
  • targetingsentencelines
  • adrules_governed
  • copies
  • keywordstats
  • leads
  • targetingsentencelines

Duration and time_increment

When to use duration?

  • The duration has effect only when incremental = true
  • The duration is used to compute time range for your request, based on last_fetched_at, for example, if last_fetched_at = 02/01/2017 and duration = 2, you’re requesting for a time range of: 02/01/2017 → 02/02/2017

Directly translated to Facebook request parameters:

.time_range({"since":"2017-02-01","until":"2017-02-02"})
  • There is no breakdown behavior when you use this parameter. It returns insight values for the whole time range.
  • Note: When no last_fetched_at is specified, it is 30 days from today, ie. (today – 30 days)

When to use time_increment?

  • time_increment is used when you want to break down insights data by a number of days. The results are easiest to recognize without incremental mode and duration.

Example:

incremental: false
last_fetched_at: 03/01/2017
last_fetched_until: 03/31/2017
time_increment: 1

Using the preceding configuration, you can import the entire month of March 2017 in one run, and still have it break down daily.

duration and time_increment differences

  • duration is used for incremental mode
  • time_increment is an official Facebook parameter
  • Important note: If both duration and time_increment are available (which is default), duration will have higher priority than time_increment when its value is less than time_increment. With that being said, you can’t have breakdown data per 2 days (time_increment = 2) when duration = 1, result will be 1-day data. In the other hand, you can perfectly break down data daily (time_increment = 1) for a 2-day duration (duration = 2).

When to use last_fetched_until?

  • last_fetched_until is optional, and its only usage is to limit imported data to an end date.

Example: you have a job to import March 2017, and it’s scheduled to run repeatedly to get daily data. Therefore, you could use the following configuration to achieve it:

incremental: true
duration: 1
last_fetched_at: 03/01/2017
last_fetched_until: 03/31/2017

Note: As you can see, you can simulate time_increment effect by using duration and run the job repeatedly, but it will cost you more requests (Request Rate Limit will occur faster).

  • If your scheduled job (eventually) exceeds configured last_fetched_until, an empty result is returned.

Data Connector schema change in the July 10 2018 release

Due to recent changes of Facebook API, Data Connector has updated.

Before the change: Some fields were added but the output schema was missing some columns because the value was empty.

After the change: Some new columns are added but might not contain values. The output schema will be combined from the input configuration and mandatory fields.

The mandatory fields and implicitly added columns are:

  • Metadata fields

    • created_time
    • effective_status
    • status
  • Insights Fields

    • date_start
    • date_stop

Invalid **fields** that might have been skipped in previous updates will now result in errors after update. See the Fields section of following links for complete references for Insights Fields and Metadata fields: