Data Connector for Facebook Page and Post Insights (BETA)

Facebook offers insight at two data types: page and post. This Data Connector enables import of Facebook Page and Post Insights into Treasure Data.

This feature is currently in beta. Contact us for further details about the private beta.

Table of Contents

Prerequisites

  • Basic knowledge of Treasure Data
  • Basic knowledge of Facebook Graph API
  • Basic knowledge of Page Insights API

Option 1: Use Web Console

Create a new connection

Go to Treasure Data Connections. Search and select Facebook Page Insights. A dialog will open.



Select an existing OAuth connection for Facebook, or click the link under OAuth connection to create a new connection.



Create a new OAuth connection

Login to your Facebook account in popup window:



And grant access to Treasure Data app.

You will be redirected back to Treasure Data Connections. Repeat the first step (Create a new connection) and choose your new OAuth connection.



Create a new transfer

Upon creating the connection, you will be automatically taken to My Connections tab. Look for the connection you created and click New Transfer.



Complete the following details and click on “Next”.

Page Name

You can find Page Name in Facebook URL, for example: https://www.facebook.com/TreasureData, the value you need is: TreasureData.

Data Type

There are 2 data_type: "page" and "post".

Metrics

By unchecking Use Preset Metrics, you can specify individual metrics.



Metric Presets

Or you can simply check Use Preset Metrics and pick predefined values from the drop-down box.



The following table lists supported values for the Metric Presets field:

Facebook’s Name For Data Type Value to Use
Page Impressions page page_impressions
Page Engagement page page_engagement
Page CTA Clicks page page_cta_clicks
Page User Demographics page page_user_demographics
Page Views page page_views
Page Posts post page_posts
Page Post Impressions post page_post_impressions
Page Post Engagement post page_post_engagement
Page Post Reactions post page_post_reactions
Page Video Posts post page_video_posts
Post Stories By Action Types post post_stories_by_action_type

Start Date and End Date

Start Date and End Date must follow format: YYYY-MM-DD, eg. 2017-11-21.

Result data will be aggregated by the end of each day, meaning insights of 2017-01-01 will have end_time = 2017-01-02 00:00:00.



Untitled-3
Facebook APIs and other Facebook interfaces vary in how Start date and End dates are defined. The Start and End date you specify while defining this connector is passed to the Facebook API that you specify. You might need to adjust the Start and End dates that you specify to ensure that, within Facebook, the various API and interface Start and End dates match.

If you use Incremental Loading and scheduled jobs, the time range for the next iteration (run) will be calculated based on the initial Start Date and End Date. Imported data will have no gaps.

Due to the way Facebook API query operates, you will need to pay attention to End Date if you’re running a one-time job.

Your result will include insights data with end_time spanning from <Start Date + 1> 00:00:00 to <End Date - 1> 00:00:00.

If you want to import the insights for the month of January 2017, the End Date must be used carefully. For example, if you chose the Start Date and End Date this way:

  • Start Date = 2017-01-01 (or 2017-01-01 00:00:00)
  • End Date = 2017-02-01 (or 2017-02-01 00:00:00)

Your output will look similar to the following example:

+---------------------------+-------------------------+--------------------------------+------------------------------+----------------------------------+------------------------------------+------------------------+-------------------------------+--------------------------------+-----------------------------------+-----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------+---------------------------+----------------------------------+---------------------------------------------------------------------+
| end_time:timestamp        | lifetime_page_fans:json | lifetime_page_fans_locale:json | lifetime_page_fans_city:json | lifetime_page_fans_country:json  | lifetime_page_fans_gender_age:json | day_page_fan_adds:json | day_page_fan_adds_unique:json | week_page_fan_adds_unique:json | days_28_page_fan_adds_unique:json | day_page_fans_by_like_source:json                                                 | day_page_fans_by_like_source_unique:json                                          | day_page_fans_group_by_like_source_unique:json                                | day_page_fan_removes:json | day_page_fan_removes_unique:json | day_page_fans_by_unlike_source_unique:json                          |
+---------------------------+-------------------------+--------------------------------+------------------------------+----------------------------------+------------------------------------+------------------------+-------------------------------+--------------------------------+-----------------------------------+-----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------+---------------------------+----------------------------------+---------------------------------------------------------------------+
| "2017-01-02 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "5"                    | "5"                           | "29"                           | "122"                             | "{\"sponsored_story\":5}"                                                         | "{\"sponsored_story\":5}"                                                         | "{\"Ads\":5}"                                                                 | "0"                       | "0"                              | nil                                                                 |
| "2017-01-03 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "7"                    | "7"                           | "32"                           | "122"                             | "{\"feed_story\":1,\"sponsored_story\":6}"                                        | "{\"feed_story\":1,\"sponsored_story\":6}"                                        | "{\"Uncategorized Desktop\":1,\"Ads\":6}"                                     | "1"                       | "1"                              | "{\"pagelike_deleter_for_deactivated_users\":1}"                    |
| ...                       | ...                     | ...                            | ...                          | ...                              | ...                                | ...                    | ...                           | ...                            | ...                               | ...                                                                               | ...                                                                               | ...                                                                           | ...                       | ...                              | ...                                                                 |
| "2017-01-30 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "7"                    | "7"                           | "30"                           | "125"                             | "{\"feed_story\":1,\"sponsored_story\":5,\"hovercard\":1}"                        | "{\"feed_story\":1,\"sponsored_story\":5,\"hovercard\":1}"                        | "{\"Your Posts\":1,\"Uncategorized Desktop\":1,\"Ads\":5}"                    | "0"                       | "0"                              | nil                                                                 |
| "2017-01-31 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "4"                    | "4"                           | "32"                           | "115"                             | "{\"feed_story\":1,\"sponsored_story\":3}"                                        | "{\"feed_story\":1,\"sponsored_story\":3}"                                        | "{\"Uncategorized Desktop\":1,\"Ads\":3}"                                     | "2"                       | "1"                              | "{\"normal_unfan\":1}"                                              |
+---------------------------+-------------------------+--------------------------------+------------------------------+----------------------------------+------------------------------------+------------------------+-------------------------------+--------------------------------+-----------------------------------+-----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------+---------------------------+----------------------------------+---------------------------------------------------------------------+

The result will contain data up to and including Jan 30th, ending with the beginning of Jan 31st, but will include no data for Jan 31st. So, in order to get data for the entire month, you must set End Date = 2017-02-02:

+---------------------------+-------------------------+--------------------------------+------------------------------+----------------------------------+------------------------------------+------------------------+-------------------------------+--------------------------------+-----------------------------------+-----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------+---------------------------+----------------------------------+---------------------------------------------------------------------+
| end_time:timestamp        | lifetime_page_fans:json | lifetime_page_fans_locale:json | lifetime_page_fans_city:json | lifetime_page_fans_country:json  | lifetime_page_fans_gender_age:json | day_page_fan_adds:json | day_page_fan_adds_unique:json | week_page_fan_adds_unique:json | days_28_page_fan_adds_unique:json | day_page_fans_by_like_source:json                                                 | day_page_fans_by_like_source_unique:json                                          | day_page_fans_group_by_like_source_unique:json                                | day_page_fan_removes:json | day_page_fan_removes_unique:json | day_page_fans_by_unlike_source_unique:json                          |
+---------------------------+-------------------------+--------------------------------+------------------------------+----------------------------------+------------------------------------+------------------------+-------------------------------+--------------------------------+-----------------------------------+-----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------+---------------------------+----------------------------------+---------------------------------------------------------------------+
| "2017-01-02 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "5"                    | "5"                           | "29"                           | "122"                             | "{\"sponsored_story\":5}"                                                         | "{\"sponsored_story\":5}"                                                         | "{\"Ads\":5}"                                                                 | "0"                       | "0"                              | nil                                                                 |
| "2017-01-03 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "7"                    | "7"                           | "32"                           | "122"                             | "{\"feed_story\":1,\"sponsored_story\":6}"                                        | "{\"feed_story\":1,\"sponsored_story\":6}"                                        | "{\"Uncategorized Desktop\":1,\"Ads\":6}"                                     | "1"                       | "1"                              | "{\"pagelike_deleter_for_deactivated_users\":1}"                    |
| ...                       | ...                     | ...                            | ...                          | ...                              | ...                                | ...                    | ...                           | ...                            | ...                               | ...                                                                               | ...                                                                               | ...                                                                           | ...                       | ...                              | ...                                                                 |
| "2017-01-31 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "7"                    | "7"                           | "30"                           | "125"                             | "{\"feed_story\":1,\"sponsored_story\":5,\"hovercard\":1}"                        | "{\"feed_story\":1,\"sponsored_story\":5,\"hovercard\":1}"                        | "{\"Your Posts\":1,\"Uncategorized Desktop\":1,\"Ads\":5}"                    | "0"                       | "0"                              | nil                                                                 |
| "2017-02-01 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "4"                    | "4"                           | "32"                           | "115"                             | "{\"feed_story\":1,\"sponsored_story\":3}"                                        | "{\"feed_story\":1,\"sponsored_story\":3}"                                        | "{\"Uncategorized Desktop\":1,\"Ads\":3}"                                     | "2"                       | "1"                              | "{\"normal_unfan\":1}"                                              |
+---------------------------+-------------------------+--------------------------------+------------------------------+----------------------------------+------------------------------------+------------------------+-------------------------------+--------------------------------+-----------------------------------+-----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------+---------------------------+----------------------------------+---------------------------------------------------------------------+


Date Preset

Instead of Start Date and End Date, you can also check Use Date Preset and pick a predefined value from the drop-down box.

Untitled-3
Checking Date Preset disables Incremental Loading.

Aggregation Period

If you want to filter import data by a specific period, select the period from the dropdown. For example, if you only want the data aggregated weekly, select week as the Aggregation Period. This will select the week_xxx metrics for transfer and leave other periods, such as day_xxx, days_28_xxx, and lifetime_xxx, will not be transferred.

The selected aggregation period must be supported by at least 1 of the metrics selected. See this link to determine the periods supported by each metric.



Incremental Loading

By enabling Incremental Loading, you can schedule a job to run iteratively. The next iteration of the job run is calculated from the Start Date and End Date values.



Next, you will see a Preview of your data similar to the following dialog. Click Next.



The third step is to select the database and table where you want to transfer the data, as shown in the following dialog:



Finally, specify the schedule of the data transfer using the following dialog and click Start Transfer:



You will see the new data transfer in progress listed under the My Input Transfers tab and a corresponding job will be listed in the Jobs section.

Now, you are ready to start analyzing your data.

Option 2: Use Command Line

Step 0: Install ‘td’ command

Install the newest Treasure Data Toolbelt.

Step 1: Obtain a Facebook Token

Facebook provides 3 types of tokens. You will need the Page Access Token. We recommend that you select the never-expiring Page Access Token.

To obtain the never-expiring Page Access Token, follow the instructions here: https://www.rocketmarketinginc.com/blog/get-never-expiring-facebook-page-access-token/

Step 2: Prepare Configuration and Preview Data

First, prepare config.yml as shown:

For Page data type

in:
  type: "facebook_page_insights"
  access_token: "[your Facebook access token]"
  page_id: [your Facebook Page ID]
  data_type: page
  page_metric_presets:
  - value: page_impressions
  - value: page_cta_clicks
  - value: page_user_demographics
  - value: page_views
  - value: page_engagement
  since: 2017-01-01
  until: 2017-01-31
out:
  mode: append

For Post data type

in:
  type: "facebook_page_insights"
  access_token: "[your Facebook access token]"
  page_id: [your Facebook Page ID]
  data_type: post
  post_metric_presets:
  - value: page_posts
  - value: page_post_impressions
  - value: page_post_engagement
  - value: page_post_reactions
  - value: page_video_posts
  - value: post_stories_by_action_type
  since: 2017-01-01
  until: 2017-01-31
out:
  mode: append

For details on available out modes, see Appendix A

Then, you can see a preview of the data using the preview command.

$ td connector:preview config.yml
+---------------------------+-------------------------+--------------------------------+------------------------------+----------------------------------+------------------------------------+------------------------+-------------------------------+--------------------------------+-----------------------------------+-----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------+---------------------------+----------------------------------+---------------------------------------------------------------------+
| end_time:timestamp        | lifetime_page_fans:json | lifetime_page_fans_locale:json | lifetime_page_fans_city:json | lifetime_page_fans_country:json  | lifetime_page_fans_gender_age:json | day_page_fan_adds:json | day_page_fan_adds_unique:json | week_page_fan_adds_unique:json | days_28_page_fan_adds_unique:json | day_page_fans_by_like_source:json                                                 | day_page_fans_by_like_source_unique:json                                          | day_page_fans_group_by_like_source_unique:json                                | day_page_fan_removes:json | day_page_fan_removes_unique:json | day_page_fans_by_unlike_source_unique:json                          |
+---------------------------+-------------------------+--------------------------------+------------------------------+----------------------------------+------------------------------------+------------------------+-------------------------------+--------------------------------+-----------------------------------+-----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------+---------------------------+----------------------------------+---------------------------------------------------------------------+
| "2017-01-11 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "5"                    | "5"                           | "29"                           | "122"                             | "{\"sponsored_story\":5}"                                                         | "{\"sponsored_story\":5}"                                                         | "{\"Ads\":5}"                                                                 | "0"                       | "0"                              | nil                                                                 |
| "2017-01-03 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "7"                    | "7"                           | "32"                           | "122"                             | "{\"feed_story\":1,\"sponsored_story\":6}"                                        | "{\"feed_story\":1,\"sponsored_story\":6}"                                        | "{\"Uncategorized Desktop\":1,\"Ads\":6}"                                     | "1"                       | "1"                              | "{\"pagelike_deleter_for_deactivated_users\":1}"                    |
| "2017-01-04 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "5"                    | "5"                           | "29"                           | "125"                             | "{\"sponsored_story\":1,\"page_profile\":1,\"page_timeline\":2,\"hovercard\":1}"  | "{\"sponsored_story\":1,\"page_profile\":1,\"page_timeline\":2,\"hovercard\":1}"  | "{\"On Your Page\":1,\"Your Posts\":1,\"Uncategorized Desktop\":2,\"Ads\":1}" | "1"                       | "1"                              | "{\"normal_unfan\":1}"                                              |
| "2017-01-10 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "6"                    | "6"                           | "29"                           | "117"                             | "{\"feed_story\":1,\"sponsored_story\":3,\"page_profile\":1,\"page_timeline\":1}" | "{\"feed_story\":1,\"sponsored_story\":3,\"page_profile\":1,\"page_timeline\":1}" | "{\"On Your Page\":1,\"Uncategorized Desktop\":2,\"Ads\":3}"                  | "0"                       | "0"                              | nil                                                                 |
| "2017-01-09 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "3"                    | "3"                           | "30"                           | "117"                             | "{\"pagelike_adder_for_reactivated_users\":1,\"page_timeline\":2}"                | "{\"pagelike_adder_for_reactivated_users\":1,\"page_timeline\":2}"                | "{\"Uncategorized Desktop\":3}"                                               | "0"                       | "0"                              | nil                                                                 |
| "2017-01-02 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "4"                    | "4"                           | "28"                           | "120"                             | "{\"sponsored_story\":3,\"timeline_like_chaining\":1}"                            | "{\"sponsored_story\":3,\"timeline_like_chaining\":1}"                            | "{\"Uncategorized Desktop\":1,\"Ads\":3}"                                     | "0"                       | "0"                              | nil                                                                 |
| "2017-01-16 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "7"                    | "7"                           | "40"                           | "118"                             | "{\"feed_story\":2,\"sponsored_story\":3,\"page_profile\":2}"                     | "{\"feed_story\":2,\"sponsored_story\":3,\"page_profile\":2}"                     | "{\"On Your Page\":2,\"Uncategorized Desktop\":2,\"Ads\":3}"                  | "4"                       | "2"                              | "{\"normal_unfan\":2}"                                              |
| "2017-01-06 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "2"                    | "2"                           | "31"                           | "117"                             | "{\"feed_story\":1,\"sponsored_story\":1}"                                        | "{\"feed_story\":1,\"sponsored_story\":1}"                                        | "{\"Uncategorized Desktop\":1,\"Ads\":1}"                                     | "0"                       | "0"                              | nil                                                                 |
| "2017-01-14 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "1"                    | "1"                           | "27"                           | "107"                             | "{\"feed_story\":1}"                                                              | "{\"feed_story\":1}"                                                              | "{\"Uncategorized Desktop\":1}"                                               | "0"                       | "0"                              | nil                                                                 |
| "2017-01-15 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "11"                   | "11"                          | "36"                           | "116"                             | "{\"feed_story\":1,\"sponsored_story\":6,\"page_profile\":3,\"search\":1}"        | "{\"feed_story\":1,\"sponsored_story\":6,\"page_profile\":3,\"search\":1}"        | "{\"On Your Page\":3,\"Search\":1,\"Uncategorized Desktop\":1,\"Ads\":6}"     | "1"                       | "1"                              | "{\"pagelike_deleter_for_deactivated_users\":1}"                    |
| "2017-01-05 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "5"                    | "5"                           | "29"                           | "120"                             | "{\"feed_story\":3,\"sponsored_story\":1,\"page_timeline\":1}"                    | "{\"feed_story\":3,\"sponsored_story\":1,\"page_timeline\":1}"                    | "{\"Uncategorized Desktop\":4,\"Ads\":1}"                                     | "2"                       | "2"                              | "{\"pagelike_deleter_for_deactivated_users\":1,\"normal_unfan\":1}" |
| "2017-01-08 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "2"                    | "2"                           | "31"                           | "120"                             | "{\"feed_story\":2}"                                                              | "{\"feed_story\":2}"                                                              | "{\"Uncategorized Desktop\":2}"                                               | "1"                       | "1"                              | "{\"normal_unfan\":1}"                                              |
| "2017-01-07 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "6"                    | "6"                           | "35"                           | "118"                             | "{\"feed_story\":1,\"sponsored_story\":2,\"page_profile\":2,\"page_timeline\":1}" | "{\"feed_story\":1,\"sponsored_story\":2,\"page_profile\":2,\"page_timeline\":1}" | "{\"On Your Page\":2,\"Uncategorized Desktop\":2,\"Ads\":2}"                  | "0"                       | "0"                              | nil                                                                 |
| "2017-01-12 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "7"                    | "7"                           | "30"                           | "125"                             | "{\"feed_story\":1,\"sponsored_story\":5,\"hovercard\":1}"                        | "{\"feed_story\":1,\"sponsored_story\":5,\"hovercard\":1}"                        | "{\"Your Posts\":1,\"Uncategorized Desktop\":1,\"Ads\":5}"                    | "0"                       | "0"                              | nil                                                                 |
| "2017-01-13 08:00:00 UTC" | "1234"                  | "{}"                           | "{}"                         | "{}"                             | "{}"                               | "4"                    | "4"                           | "32"                           | "115"                             | "{\"feed_story\":1,\"sponsored_story\":3}"                                        | "{\"feed_story\":1,\"sponsored_story\":3}"                                        | "{\"Uncategorized Desktop\":1,\"Ads\":3}"                                     | "2"                       | "1"                              | "{\"normal_unfan\":1}"                                              |
+---------------------------+-------------------------+--------------------------------+------------------------------+----------------------------------+------------------------------------+------------------------+-------------------------------+--------------------------------+-----------------------------------+-----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------+-------------------------------------------------------------------------------+---------------------------+----------------------------------+---------------------------------------------------------------------+
15 rows in set
Untitled-3
You do not need to run td connector:guess for the Facebook Page Insights connector.

Step 3: Execute Load Job

Finally, submit the load job. It may take a couple of hours to process depending on the size of the data.

The Facebook Page Insights connector provides thetime column automatically.

$ td connector:issue config.yml --database td_sample_db --table td_sample_table

You can also use the --time-column option to override auto-generated time values, by specifying end_time as the time column (only applied for data_type page). Data will be accumulated daily and end_time will be end of the day, using the timezone of your Facebook Page, but converted to UTC format.

$ td connector:issue config.yml --database td_sample_db --table
td_sample_table --time-column end_time

Scheduled Execution

You can schedule periodic data connector execution for incremental Facebook Insights data. We take great care in distributing and operating our scheduler in order to achieve high availability. By using this feature, you no longer need a cron daemon on your local datacenter.

For the scheduled import, at first run, the Data Connector for Facebook Page Insights imports all of your ad data.

On the second and subsequent runs, the connector imports only data that is newer than the last load.

Create the schedule

A new schedule can be created using the td connector:create command. The following are required: the name of the schedule, the cron-style schedule, the database and table where their data will be stored, and the data connector configuration file.

$ td connector:create \
    daily_import \
    "10 0 * * *" \
    td_sample_db \
    td_sample_table \
    config.yml
Untitled-3
The `cron` parameter also accepts three special options: `@hourly`, `@daily` and `@monthly`.
Untitled-3
By default, the schedule is setup in UTC timezone. You can set the schedule in a timezone using -t or --timezone option. Note that the `--timezone` option supports only extended timezone formats like 'Asia/Tokyo', 'America/Los_Angeles' etc. Timezone abbreviations like PST, CST are *not* supported and may lead to unexpected schedules.

List All Schedules

You can see the list of all current schedule entries with the command td connector:list.

$ td connector:list
+--------------+------------+----------+-------+--------------+-----------------+----------------------------+
| Name         | Cron       | Timezone | Delay | Database     | Table           | Config                     |
+--------------+------------+----------+-------+--------------+-----------------+----------------------------+
| daily_import | 10 0 * * * | UTC      | 0     | td_sample_db | td_sample_table | {"in"=>{"type"=>"fac", ... |
+--------------+------------+----------+-------+--------------+-----------------+----------------------------+

Show Schedule Settings And History

td connector:show shows the execution settings of a schedule entry.

% td connector:show daily_import
Name     : daily_import
Cron     : 10 0 * * *
Timezone : UTC
Delay    : 0
Database : td_sample_db
Table    : td_sample_table
Config
---
in:
  type: "facebook_page_insights"
  access_token: "[your Facebook access token]"
  page_id: [your Facebook Page ID]
  data_type: page
  metric_presets:
  - page_impressions
  date_preset: last_7d
out:
  mode: append

td connector:history shows the execution history of a schedule entry. To investigate the results of each individual run, use td job <jobid>.

% td connector:history daily_import
+--------+---------+---------+--------------+-----------------+----------+---------------------------+----------+
| JobID  | Status  | Records | Database     | Table           | Priority | Started                   | Duration |
+--------+---------+---------+--------------+-----------------+----------+---------------------------+----------+
| 578066 | success | 10000   | td_sample_db | td_sample_table | 0        | 2015-04-18 00:10:05 +0000 | 160      |
| 577968 | success | 10000   | td_sample_db | td_sample_table | 0        | 2015-04-17 00:10:07 +0000 | 161      |
| 577914 | success | 10000   | td_sample_db | td_sample_table | 0        | 2015-04-16 00:10:03 +0000 | 152      |
| 577872 | success | 10000   | td_sample_db | td_sample_table | 0        | 2015-04-15 00:10:04 +0000 | 163      |
| 577810 | success | 10000   | td_sample_db | td_sample_table | 0        | 2015-04-14 00:10:04 +0000 | 164      |
| 577766 | success | 10000   | td_sample_db | td_sample_table | 0        | 2015-04-13 00:10:04 +0000 | 155      |
| 577710 | success | 10000   | td_sample_db | td_sample_table | 0        | 2015-04-12 00:10:05 +0000 | 156      |
| 577610 | success | 10000   | td_sample_db | td_sample_table | 0        | 2015-04-11 00:10:04 +0000 | 157      |
+--------+---------+---------+--------------+-----------------+----------+---------------------------+----------+
8 rows in set

Delete Schedule

td connector:delete removes the schedule.

$ td connector:delete daily_import

Configuration

The following table provides details on the options available in modes.

Option name Description Type Required? Default Value
access_token Facebook Page Access Token (see Obtain Facebook Token section) string yes
page_id Facebook Page ID string yes
data_type "page" or "post" string optional "page"
metric_presets Predefined category of metrics, or group of related metrics. See Appendix B for available values array optional
metrics Facebook Graph insight metrics, you can specify each metric, as much as you need. This config will override metric_preset if both of them are specified. Available metrics are here array optional
since Lower bound of the time range to consider, supported formats: yyyy-MM-dd or Epoch timestamp without milliseconds string optional
until Upper bound of the time range to consider, supported formats: yyyy-MM-dd or Epoch timestamp without milliseconds string optional
period The aggregation period. See Appendix C for available values enum optional
date_preset Preset a date range, like ‘lastweek’ or ‘yesterday’. If a ‘since’ or ‘until’ date is specified and the date_preset is also selected, the data transfer request will fail. See Appendix D for available values enum optional
incremental true for generate “config_diff” with embulk run -c config.diff bool optional false
retry_limit Number of error retries before connector gives up integer optional 7
retry_initial_wait_millis Wait milliseconds for exponential backoff initial value integer optional 500 (0.5 second)
max_retry_wait_millis Maximum wait milliseconds for each retry integer optional 300000 (5 minutes)
connect_timeout_millis HTTP connect timeout in milliseconds integer optional 180000 (3 minutes)
idle_timeout_millis HTTP idle timeout in milliseconds integer optional 300000 (5 minutes)

Appendix

Appendix A: Modes for out plugin

You can specify file import mode in out section of seed.yml.

append (default)

This is the default mode and records are appended to the target table.

in:
  ...
out:
  mode: append

replace (In td 0.11.10 and later)

This mode replaces data in the target table. Any manual schema changes made to the target table remains intact with this mode.

in:
  ...
out:
  mode: replace

Appendix B: Supported values of metric_presets

  • page_impressions
  • page_cta_clicks
  • page_user_demographics
  • page_views
  • page_engagement
  • page_posts
  • page_post_impressions
  • page_post_engagement
  • page_post_reactions
  • page_video_posts
  • post_stories_by_action_type

Appendix C: Supported values of period

  • day
  • week
  • days_28
  • month
  • lifetime
  • total_over_range

Appendix D: Supported values of date_preset

  • today
  • yesterday
  • this_month
  • last_month
  • this_quarter
  • lifetime
  • last_3d
  • last_7d
  • last_14d
  • last_28d
  • last_30d
  • last_90d
  • last_week_mon_sun
  • last_week_sun_sat
  • last_quarter
  • last_year
  • this_week_mon_today
  • this_week_sun_today
  • this_year

Addendum

Page ID

Instead of using Page Name when creating the connector, you can use Page ID. To find the page ID, on your Facebook page, select the About menu and scroll down to the Page ID, as shown:




Last modified: Jan 22 2018 19:33:59 UTC

If this article is incorrect or outdated, or omits critical information, let us know. For all other issues, access our support channels.