Marketo Import Connection Using CLI

Copy

• Copy for LLM

  Copy page as Markdown for LLMs
• View as Markdown

  Open this page as Markdown
• Open in ChatGPT

  Get insights from ChatGPT
• Open in Claude

  Get insights from Claude
• Connect to Cursor

  Install MCP server on Cursor
• Connect to VS Code

  Install MCP server on VS Code

You can use the CLI to configure your connection.

This topic includes:

Limitations

• Updating the connector configuration using td connector:update with -c, --config CONFIG_FILE option is not supported.

Install the Treasure Data Toolbelt

Open a terminal and run the following command to install the newest Treasure Data Toolbelt.

$ td --version
0.11.10

Guess and Preview are supported for Leads within List, Leads within Program.

Create a Configuration File (seed.yml)

Prepare seed.yml as shown in the following example, with your parameters

in:
  type: marketo
  <parameter: value>
out:
  <mode: append

This configuration dumps the Marketo object specified in the target field because “replace” mode is specified.

For more details on available out modes, see the Appendix.

Guess Fields (Generate load.yml)

Use connector:guess. This command automatically reads the target file and assesses(uses logic to guess) the file format. and output to load.yml. The file load.yml will include a schema for Lead.

td connector:guess seed.yml -o load.yml

If you open up load.yml, you’ll see assessed file format definitions including, in some cases, file formats, encodings, column names, and types.

Then you can preview how the system parses the file by using preview command.

td connector:preview load.yml

If the system detects your column name or type incorrectly, modify load.yml directly and preview again.

The Data Connector supports parsing of "boolean", "long", "double", "string", and "timestamp" types.

Execute Load Job

Submit the load job. It may take a couple of hours depending on the data size. Users need to specify the database and table where their data is stored.

td connector:issue load.yml \
--database td_sample_db \
--table td_sample_table \
--time-column activity_date_time

The connector:issue command assumes you have already created a database(td_sample_db) and a table(td_sample_table). If the database or the table do not exist in TD, this command will fail, so create the database and table manually or use --auto-create-table option with td connector:issue command to auto-create the database and table:

td connector:issue load.yml \
--database td_sample_db \
--table td_sample_table \
--time-column activity_date_time \
--auto-create-table

You can assign a Time Format column to the "Partitioning Key" by using the "--time-column" option.

Scheduled Execution

You can schedule periodic data connector execution for periodic Marketo import. We configure our scheduler carefully to ensure high availability. By using this feature, you no longer need a cron daemon on your local data center.

For the scheduled import, Data Connector for Marketo imports all records.

Create the Schedule

A new schedule can be created by using the td connector:create command. The name of the schedule, cron-style schedule, the database and table where their data will be stored, and the Data Connector configuration file are required.

$ td connector:create \
    daily_marketo_leads_import \
    "10 0 * * *" \
    td_sample_db \
    td_sample_table \
    load.yml

The cron parameter also accepts three special options: @hourly, @daily and @monthly. For more detail on Scheduled Jobs.

By default, schedule is setup in UTC timezone. You can set the schedule in a timezone using -t or —timezone option. The --timezone option supports only extended timezone formats like ‘Asia/Tokyo’, ‘America/Los_Angeles’ etc. Timezone abbreviations like PST, CST are not supported and may lead to unexpected schedules.

List the Schedules

You can see the list of scheduled entries by td connector:list.

$ td connector:list
+-----------------------------------+---------------+----------+-------+-----------------+----------------------+
| Name                              | Cron          | Timezone | Delay | Database        | Table                |
+-----------------------------------+---------------+----------+-------+-----------------+----------------------+
| daily_marketo_leads_import.       | 10 0 * * *    | UTC      | 0     | td_sample_table | sample_table         |
+-----------------------------------+---------------+----------+-------+-----------------+----------------------+

Show the Setting and History of Schedules

td connector:show shows the execution setting of a schedule entry.

% td connector:show daily_marketo_leads_import

td connector:history shows the execution history of a schedule entry. To investigate the results of each individual execution, use td job jobid.

td connector:history daily_marketo_leads_import

Delete the Schedule

td connector:delete removes the schedule.

$ td connector:delete daily_marketo_leads_import

Modes for Out Plugin

You can specify file import mode in out the section of seed.yml.

append (default)

This is the default mode and records are appended to the target table.

in:
  ...
out:
  mode: append

replace (In td 0.11.10 and later)

This mode replaces data in the target table. Any manual schema changes made to the target table remain intact with this mode.

in:
  ...
out:
  mode: replace