Learn more about Marketo Export Integration.

You can import Marketo data into Treasure Data. Then, in Treasure Data, you can integrate the data with your other data sources.

Bulk extracts source are imported using Marketo Bulk Extract API.

For sample workflows on importing Marketo data, view Treasure Boxes

This topic includes:

You can also Use Command Line to Create Your Marketo Connection.

Prerequisites, Limitations, and Requirements


Obtain Marketo Munchkin Account Information

Access to your API-enabled account is required before you can access the API. The Munchkin Account ID can be retrieved from the Marketo Admin page.

Steps to enable and create a new account for API access

  1. Create an API Only User.

  2. Create an API Only User Role.

  3. Get Custom Object API Name, field API Name.



Use the TD Console to Create Your Connection

You can import Marketo data from TD Console.

Create a New Connection

  1. Open TD Console.

  2. Navigate to Integrations Hub > Catalog

  3. Search for and select Marketo.

    The following dialog opens.


  4. Complete the new connection information, providing your Marketo credential.

  • Marketo Account ID (required): This is your Marketo Service/Munchkin ID.

  • Marketo Client ID (required): This is service specific client id.

  • Marketo Client Secret (required): This is service specific client secret.

  1. Type a name for connection.


  2. Select Done to place your new authentication.

Transfer Your Marketo Data to Treasure Data

After creating the authenticated connection, you are automatically taken to Authentications.

  1. Search for the connection authentication that you created. 

  2. Select New Source.

Connection

  1. Type a name in the Data Transfer field.


  2. Select Next

Source Table

  1. Choose one of the following sources:
    Lead, Activity, Lead by Static List, Leads by Program, Campaign, Programs and Program Members.

  2. Type values for the following common parameters depending on the source type you have selected:


Parameters

Description



Incremental

Bulk extract sources support the incremental import feature through the createdAt filter. When running in incremental mode, the connector calculates the next ‘from_date’ based on the previous filter.

Example:

  • 1st run from_date: 2018-01-01 to_date: 2018-01-11(generated by plugin) fetch_days: 10

  • 2nd run from_date: 2018-01-11 to_date: 2018-01-21 fetch_days: 10

Limitation: – Bulk extract source preview shows only mock data.

From Date

Data that has a createdAt or updatedAt filter after the specified date.

Fetch Days

The To date is calculated using From day + Fetch days

Escape Character

Marketo CSV file escape character.

Quote Character

Marketo CSV file quote character.



Retrieve large amounts of activity data out of Marketo.

Activity attributes are imported to the column attributes. This column is in JSON format.

Parameters

Description

ACTIVITY TYPE IDS

Integer values, corresponding to the desired activity types.

Incremental

Bulk extract sources support the incremental import feature through the createdAt filter. When running in incremental mode, the connector calculates the next ‘from_date’ based on the previous filter.

Example:

  • 1st run from_date: 2018-01-01 to_date: 2018-01-11(generated by plugin) fetch_days: 10

  • 2nd run from_date: 2018-01-11 to_date: 2018-01-21 fetch_days: 10

Limitation: – Bulk extract source preview shows only mock data.

From Date

Data that has a createdAt or updatedAt filter after the specified date.

Fetch Days

The To date is calculated using From day + Fetch days.

Escape Character

Marketo CSV file escape character.

Quote Character

Marketo CSV file quote character.



Import Marketo Campaign data. There are no other fields to fill out



Parameter

Description

List IDs

Retrieves person records that are members of the given static list. Results include the column listId that contains the lead list ID.

Optional. Comma separated List IDs, or leave the field blank to import all lists.



Import Marketo leads by program data.

Parameter

Description

Program IDs

Optional. Comma separated program IDs, or leave the field blank to import all programs.



Import Marketo program data. Retrieves the list of accessible programs from the target instance.

Parameter

Description

Incremental

Bulk extract sources support the incremental import feature through the createdAt filter. When running in incremental mode, the connector calculates the next ‘from_date’ based on the previous filter.

Example:

  • 1st run from_date: 2018-01-01 to_date: 2018-01-11(generated by plugin) fetch_days: 10

  • 2nd run from_date: 2018-01-11 to_date: 2018-01-21 fetch_days: 10

Limitation: – Bulk extract source preview shows only mock data.

Query By

Query Programs by Tag Type or Date Range. Leave blank to import all programs.

With Query By: Tag Type, Incremental import is not supported, and therefore is omitted.

With Query By Date Range and Incremental import is selected, the next run Earliest Updated At and Latest Updated At will be calculated based on previous range, example:

  • 1st run: Earliest Updated At = 02/01/2018 10:00, Latest Updated At = 02/10/2018 10:00

  • 2nd run: Earliest Updated At = 02/10/2018 10:01, Latest Updated At = 02/20/2018 10:01



Retrieves a list of custom objects records based on filter and a set of values. See Custom Objects API document to learn more about configuration keys.

Parameter

Description

API Name

The API name of the custom object

Fields

Comma separated API name of fields of the custom object (Optional).

Filter Type

The API name of field of the custom object use to filter result. Only support integer field.

Input as Text

Input filter values as text (comma separated), or by range (From Value to To Value).

Filter Values

Comma-separated list of field values to match.

From Value

Filter start value. Only supports integer.

To Value

Filter end value (Optional). Only supports integer. If not set, only records that have value greater than "From Value" will be returned. Job will stop if no record found in 300 consecutive value.



Import Marketo members by program data.

Parameter

Description

Program IDs

Optional. Comma separated program IDs, or leave the field blank to import members of all programs.


Data Settings

  1. Select Next.
    The Data Settings page opens.

  2. Optionally, edit the data settings or skip this page.


Parameters

Description

LEAD INCLUDED FIELDS

Add a list of Lead fields to be included in data import, only affects Lead family target.

 Interval limit in milliseconds

Time wait for the next call if the request reaches Marketo concurrent limit.

 Maximum retries

Maximum times to retry Marketo request when error occurs.

 Batch size

Marketo REST API Batch size (default 300).

 Max return

Max records return in a single request. Program endpoint use offset for paging (default 200).

Bulk job timeout in seconds

Total time wait for bulk extract before fail the job.

Bulk job polling interval in seconds

Interval to poll job status.

Read timeout in milliseconds

Time to wait for Marketo response.



Data Preview 

You can see a preview of your data before running the import by selecting Generate Preview.

Data shown in the data preview is approximated from your source. It is not the actual data that is imported.

  1. Select Next.
    Data preview is optional and you can safely skip to the next page of the dialog if you want.

  2. To preview your data, select Generate Preview. Optionally, select Next

  3. Verify that the data looks approximately like you expect it to.


  4. Select Next.

Data Placement

Specify where your data will be placed and schedule how often it will run this import.

Under Storage you create a new or select an existing database and create a new or select an existing table for where you want to place the imported data.

  1. Select a Database > Select an existing or Create New Database.

  2. Optionally, type a database name.

  3. Select a Table> Select an existing or Create New Table.

  4. Optionally, type a table name.

  5. Choose the Append or Replace method for importing the data.

  6. Select the Timestamp-based Partition Key column.
    If you want to set a different partition key seed than the default key, you can specify the long or timestamp column as the partitioning time. As a default time column, it uses upload_time with the add_time filter.

  7. Select the Timezone for your data storage.

  8. Under Schedule, you can choose when and how often you want to run this query.

After your transfer has run, you can see the results of your transfer in Data Workbench > Databases.