Skip to content
Last updated

Google Ads Import Integration V2

Google Ads is an online advertising platform that allows businesses to promote their products and services through digital ads across millions of Google partner websites and its services such as Google Search, YouTube, and Maps. This integration allows you to import Google Ads reports and metrics into Treasure Data to tailor-made your customer experiences, make data-driven decisions to optimize your campaigns, and achieve better marketing results

Prerequisites

Requirements and Limitations

  • The import job could fail if the data is too big. Using paging could fix that problem.
  • Google might change the default views of its performance data from time to time, thus the below pre-defined metric collection of each report type might not be the same as the corresponding reports found on the Google Ads. However, missing metrics could be added to fulfill the data requirement.
  • This article doesn't intend to list all available attributes, segments, metrics, and explanations for them. For more details, please refer to Google Ads Query Builder, or the Report Editor feature which can be found on the Google Ads UI (Campaigns > Insights and reports > Report Editor)

Import from Google Ads via TD Console

You must create and configure the data connection on TD Console before importing data. As part of the data connection, you provide the OAuth authentication of your Google Ads account to deploy the integration using the following steps.

  1. Open TD Console
  2. Navigate to Integrations Hub > Catalog
  3. Search for and select Google Ads V2
  4. Select Create Authenticationand provide the OAuth of a Google account with access to the target Google Ads account
  5. After being redirected back to the Integration Hub, repeat the above steps (this time your provided OAuth will be shown), enter a name for your authentication, and select Done.

Create a Source

  1. Open TD Console
  2. Navigate to Integrations Hub > Authentications
  3. Locate your new authentication and select New Source
  4. On the first step **1 - Connection**, type a source name in the Data Transfer Name field, and select Next

Identify a Source Table

Identify what data to be ingested into Treasure Data using the parameters below, and select Next.

This integration supports flexible ways to define your target data:

  • Choose one of the 4 pre-defined report types, and pick up the desired Attributes, Segments, and Metrics. Or,
  • Fill in a custom query valid for Google Ads to import necessary data. It is recommended to use this option together with the Google Ads Query Builder

Source Table Parameter

ParameterRequiredTypeDescription
Ads AccountRequiredstringThe ID of the target Google Ads account, with or without dashes
Ads Manager AccountOptionalstringThe ID of the Google Manager account. This is required if the Ads Account is accessible via a manager account.
Enable Custom QueryRequiredbooleanWith this option enabled, a custom query can be used to declare the target data source. Refer to Google Ads Query Builder for more details. Otherwise, choose one of the pre-defined report types, and pick up the desired Attributes, Segments, and Metrics.
Report TypeOptionalstringRequired if the Custom Query option is disabled. Each pre-defined report type will have its collections of Attributes, Segments, and Metrics. - Ad Performance (similar to the dashboard found on Campaigns > Campaigns > Ads) - Ad Group Performance (similar to the dashboard found on Campaigns > Campaigns > Ad groups) - Campaign Performance (similar to the dashboard found on Campaigns > Campaigns > Campaigns) - Keywords Performance (similar to the dashboard found on Campaigns > Audiences, keywords and content > Search keywords)
Include Negative KeywordsOptionalbooleanEnable negative keywords to ingest when the report type is Keywords Performance Report
Use predefined metricsOptionalbooleanUse predefined metrics of the selected report type, instead of adding additional metrics
AttributesOptionalarray of stringList of additional attributes for the selected report type
SegmentsOptionalarray of stringAt least 1 segment must be added for all the predefined report types. Only time segment is supported: Date, Day of week, Week, Month, Quarter, Year. When the Date Range is All Time, no time segment is required. Otherwise, one type of time segment is needed.
MetricsOptionalarray of stringList of additional metrics for the selected report type
Date RangeOptionalstringThe date range of the target data. Supported values: - Custom Date: define your target range using Start Date and End Date - Today, Yesterday - Last 7 days, Last business week - This month, Last month - All time - Last 14 days, Last 30 days - This week from the previous Sunday to today - This week from the previous Monday to today - Last week from the previous Sunday to Saturday
Start DateOptionaldateThe start date of the custom date range in yyyy-MM-dd format
End DateOptionaldateThe end date of the custom date range in yyyy-MM-dd format
Incremental modeOptionalbooleanEnable to run the job in incremental mode

Source Table Parameter - Custom Query

ParameterRequiredTypeDescription
Select ColumnsOptionalstringA comma-separated list of fields to query. It should include necessary attributes, segments, and metrics.
From TargetOptionalstringThe target resource name. It will be used in the FROM clause of the query statement.
Other ConditionsOptionalstringOther conditions for the query. It will be used in the WHERE clause of the query statement.
Enable Date RangeOptionalbooleanEnable query by date range option. Disabling this option requires that Other Conditions starts with the WHERE keyword. Disabling this option also disables Incremental mode, Start Date, and End Date.

Define Data Settings

Optionally, you can modify advanced data settings. Select Next to move to the next step.

ParameterRequiredTypeDescription
Retry LimitRequiredintInternal maximum retries limit. Default: 5 times.
Initial retry time wait in millisRequiredintInitial retry waiting time in milliseconds. Default: 500ms.
Max retry wait in millisRequiredintMaximum retry waiting time in milliseconds. Default: 300,000ms.

Preview Your Data

  • On the **4 - Data Preview** step, select Generate Preview to see an approximated view of your data before running the import (optional)
  • Select Next to continue

Data Placement

For data placement, select the target database and table where you want your data placed and indicate how often the import should run.

  1. Select Next. Under Storage, you will create a new or select an existing database and create a new or select an existing table for where you want to place the imported data.

  2. Select a Database > Select an existing or Create New Database.

  3. Optionally, type a database name.

  4. Select a TableSelect an existing or Create New Table.

  5. Optionally, type a table name.

  6. Choose the method for importing the data.

    • Append (default)-Data import results are appended to the table. If the table does not exist, it will be created.
    • Always Replace-Replaces the entire content of an existing table with the result output of the query. If the table does not exist, a new table is created.
    • Replace on New Data-Only replace the entire content of an existing table with the result output when there is new data.
  7. Select the Timestamp-based Partition Key column. If you want to set a different partition key seed than the default key, you can specify the long or timestamp column as the partitioning time. As a default time column, it uses upload_time with the add_time filter.

  8. Select the Timezone for your data storage.

  9. Under Schedule, you can choose when and how often you want to run this query.

Run once

  1. Select Off.
  2. Select Scheduling Timezone.
  3. Select Create & Run Now.

Repeat Regularly

  1. Select On.
  2. Select the Schedule. The UI provides these four options: @hourly@daily and @monthly or custom cron.
  3. You can also select Delay Transfer and add a delay of execution time.
  4. Select Scheduling Timezone.
  5. Select Create & Run Now.

After your transfer has run, you can see the results of your transfer in Data Workbench > Databases.

Import from Google Ads via CLI (Toolbelt)

Prerequisites:

  • TD Toolbelt: install the latest version of the TD CLI tool: TD Toolbelt
  • Authentication ID: follow the steps above to create an Authentication for this integration on the TD Console. Then the ID could be seen at the last portion of its URL on the Authentication screen.

General steps:

  1. Create a YML configuration file, ex: **load.yml**, referencing the created authentication in the `td_authentication_id' field. Refer to the below Parameter Reference and Example for more details
  2. Preview the input data (optional) using the command: $ **td connector:preview** load.yml
  3. Trigger the data import using the command: $ **td** **connector:issue** load.yml --database db-name --table table-name
  4. Schedule the execution using the command: $ **td** **connector:create** daily\_import "10 0 \* \* \*" db-name table-name load.yml

Refer to this page for more reference information

Parameters Reference

NameDescriptionValueDefault ValueRequired
client_customer_idClient customer IDStringTrue
login_customer_idAds Manager Account ID. This is required if the Ads Account is accessible via a Google Manager account.StringFalse
targetPredefined report type. Used when enable\_custom\_query is false.String- ad_performance_report - adgroup_performance_report - campaign_performance_report - keywords_performance_reportFalse
segmentsList of additional segments.  Used when enable\_custom\_query is false. When the date\_range is ALL_TIME, no time segment is required Otherwise, one type of time segment is needed. time segment is supported: Date, Day of week, Week, Month, Quarter, YearArray of StringFalse
include_predefined_metricsInclude all predefined metrics Used when enable\_custom\_query is false.BooleanTrueFalse
metricsList of additional metrics. Used when enable\_custom\_query is false.Array of StringFalse
attributesList of additional attributes. Used when enable\_custom\_query is falseArray of StringFalse
incrementalRun the job in incremental modeBooleanFalseFalse
incremental_durationDuration in days for the incremental job. Ignore end_date and recalculate it by this formula:   end_date = start_date + incremental_durationInteger0False
data_rangeThe date range of the target data. Refer to here for supported valuesString
start_dateStart date, used with date\_range is custom\_dateDateFalse
end_dateEnd date, used with date\_range is custom\_dateDateFalse
include_negative_keywordsFilter the records by negative keywords, only used with keywords_performance_reportFalseFalseFalse
enable_custom_queryEnable the use of custom queryBooleanFalseTrue
select_columnsList of fields to query, separated by commaStringFalse
from_targetReport target nameStringFalse
other_conditionsThe other condition of the queryStringFalse
enable_date_rageEnable data range query and incremental modeBooleanTrueFalse
td_authentication_idThe ID of an Authentication created on the TD ConsoleStringEnable query by date range option. Disable this option will require Other Conditions to start with WHERE keyword. Disable this option also disable Incremental mode, Start Date, and End Date alsoTrue
refresh_tokenThe authentication info is required, and it is more convenient to create it on the UI and then refer to its ID using the above parameter. Alternatively, you can create a Google API project to provide the credential information required for these four parameters.StringFalse
client_idStringFalse
client_secretStringFalse

Examples

Disable custom query

in:
  type: google_adwords_v2
  td_authentication_id: 330392
  enable_custom_query: false
  client_customer_id: xx-xxxx-xxxx
  target: AD_PERFORMANCE_REPORT
  date_range: "CUSTOM_DATE"
  include_predefined_metrics: false
  incremental: true
  start_date: 2020-03-01
  end_date: 2020-03-02
  segments: ["segments.date"]
  metrics: ["metrics.absolute_top_impression_percentage"]
out:
  mode: append

Enable custom query

in:
  type: google_adwords_v2
  td_authentication_id: 330392
  enable_custom_query: true
  client_customer_id: xx-xxxx-xxxx
  date_range: "CUSTOM_DATE"
  enable_date_range: true
  incremental: true
  start_date: 2020-03-01
  end_date: 2020-03-02
  select_columns: ad_group_criterion.criterion_id, ad_group.id, ad_group.name, segments.date
  from_target: keyword_view
  other_conditions: "AND ad_group_criterion.type = 'KEYWORD'"
out:
  mode: append

Use with Manager Ads Account

in:
  type: google_adwords_v2
  td_authentication_id: 330392
  enable_custom_query: true
  client_customer_id: xx-xxxx-xxxx
  login_customer_id: xx-xxxx-xxxx
  date_range: "CUSTOM_DATE"
  incremental: true
  start_date: 2020-03-01
  end_date: 2020-03-02
  select_columns: ad_group_criterion.criterion_id, ad_group.id, ad_group.name, segments.date
  from_target: keyword_view
  other_conditions: "AND ad_group_criterion.type = 'KEYWORD'"
out:
  mode: append

Import from Google Ads V2 via Workflow

You can import data from Google Ads by using the TD Workflow. Refer here for more detailed information.

How to convert Google Ads Query Language to a custom query in the connector

Query language reference: https://developers.google.com/google-ads/api/docs/query/overview?hl=en

Example:

SELECT
  campaign.id,
  campaign.name,
  campaign.status,
  metrics.impressions,
  segments.date,
FROM campaign
WHERE segments.date during LAST_30_DAYS
  AND campaign.status = 'PAUSED'
  AND metrics.impressions > 1000
ORDER BY campaign.id
in:        type: google_adwords_v2  enable_custom_query: true  client_id: xxx  client_secret: xxx  refresh_token: xxx  client_customer_id: xx-xxxx-xxxx  date_range: "LAST_30_DAYS"  developer_token: xxx  select_columns: campaign.id, campaign.name, campaign.status, metrics.impressions, segments.date  from_target: campaign  other_conditions: "AND campaign.status = 'PAUSED' AND metrics.impressions > 1000"
in:        type: google_adwords_v2  enable_custom_query: true  client_id: xxx  client_secret: xxx  refresh_token: xxx  client_customer_id: xx-xxxx-xxxx  login_customer_id: xx-xxxx-xxxx  date_range: "LAST_30_DAYS"  developer_token: xxx  select_columns: campaign.id, campaign.name, campaign.status, metrics.impressions, segments.date  from_target: campaign  other_conditions: "AND campaign.status = 'PAUSED' AND metrics.impressions > 1000"

Note: The query should have time and it should present the where condition, other_conditions  should start with AND as in the example.

Predefined Metrics

Report TypePredefined Metrics
Ad Performance AdGroup Performance Campaign Performance
  • Clicks
  • Impressions
  • Ctr
  • AverageCpc
  • Cost
  • Conversions
  • ViewThroughConversions
  • CostPerConversion
  • ConversionRate
  • ConversionFromInteractionsRate
Keywords Performance
  • Clicks
  • Impressions
  • Ctr
  • AverageCpc
  • Cost
  • Conversions
  • ViewThroughConversions
  • CostPerConversion
  • ConversionRate
  • AbsoluteTopImpressionPercentage
  • TopImpressionPercentage

Available Date Ranges

Date RangeDescription
TODAYToday only.
YESTERDAYYesterday only.
LAST_7_DAYSThe last 7 days not including today.
LAST_WEEKThe seven-day period starting with previous Monday.
LAST_BUSINESS_WEEKThe 5-day business week, Monday through Friday, of the previous business week.
THIS_MONTHAll days in the current month.
LAST_MONTHAll days in the previous month.
ALL_TIMEThe entire available time range.
CUSTOM_DATEA custom date range. Need start_date and end_date in yyyy-MM-dd.
LAST_14_DAYSThe last 14 days not including today.
LAST_30_DAYSThe last 30 days not including today.
THIS_WEEK_SUN_TODAYThe period between the previous Sunday and the current day.
THIS_WEEK_MON_TODAYThe period between the previous Monday and the current day.
LAST_WEEK_SUN_SATThe seven-day period starting with the previous Sunday.