Skip to content
Last updated

Klaviyo Import Integration

Klaviyo is a digital marketing platform focused on email marketing automation for e-commerce businesses. Klaviyo offers an SMS and email marketing automation platform for e-commerce, and its free account allows for 250 contacts and sending 500 emails.

This TD import allows you to transfer data from your Klaviyo account to Treasure Data.

Prerequisites

  • Basic knowledge of Treasure Data, including TD Toolbelt
  • An account in Klaviyo
  • A private API key for your Klaviyo account

Requirements and Limitations

Static IP Address of Treasure Data

The static IP address of Treasure Data is the access point and source of the linkage for this Integration. To determine the static IP address, contact your Customer Success representative or Technical support.

Obtain your Klaviyo API Key

To import from Klaviyo, you will need to have an API key associated with your Klaviyo account.

  1. Using your Klaviyo credentials, login to the https://www.klaviyo.com/login.
  2. Select Settings.

  1. Select the Account tab, and then select Create Private API Key.

  1. Enter the API Key name and select access scopes for your API Key.

  1. Select Create.

Import from Klaviyo via TD Console

Create Authentication

Your first step is to create a new authentication with a set of credentials.

  1. Select Integrations Hub > Catalog.

  1. Search for Klaviyo in the Catalog; hover your mouse over the icon and select Create Authentication.

  1. Ensure that the Credentials tab is selected and then enter your Klaviyo Private API key.
  2. Select Continue.
  3. Enter a name for your authentication, and select Done.

Create a Source

  1. Open TD Console.
  2. Navigate to Integrations Hub > Authentications.
  3. Locate the Klaviyo authentication and select New Source.

Create a Connection

ParameterDescription
Data Transfer NameYou can define the name of your transfer.
AuthenticationThe authentication name that is used to a transfer.
  1. Type a source name in the Data Transfer Name field.
  2. Select Next.

The Create Source page displays with the Source Table tab selected.

Identify a Source Table

The following table describes the parameters for configuring a source table:

ParameterMandatoryDescription
Data SourceYesSpecify which data we need to Ingest. Supported values:
  1. Metric Data: Get all metrics in an account.
  2. Events: Get all events in an account
  3. Event Metrics: Get the metric for an event with the given event ID.
  4. Event Profiles: Get the profile associated with an event with the given event ID.
  5. Email Templates: Get all templates in an account.
  6. Campaigns: Gets some or all campaigns
  7. Lists: Get all lists in an account.
  8. Profiles in List: Get all profiles within a list with the given list ID.
  9. Profiles: Get all profiles in an account.
RevisionNoAPI endpoint revision (format: YYYY-MM-DD). For more details, refer to section Appendix A. Recommend stable revision values:
  • 2022-10-17
  • 2023-01-24
  • 2023-02-22
Skip Related ObjectNoThis checkbox is only visible if the Data Source is Events. To skip getting details on metric and profile objects, the user can get those by making separate configurations and then joining those tables later.

Incremental

No

This checkbox is only visible if the Data Source is one of these values:

  • Events
  • Email Templates
  • Campaigns
  • Lists
  • Profiles

Selecting the checkbox enables incremental data loading with a custom start time-end time.

Filter By

No

Select which object attribute would be used in time-based filter using the start time and end time:

  • Updated
  • Created

The availability of this option depends on the data source.

Start Time - End TimeNoRequest data from Klaviyo for this period. Format: ISO 8601 format (yyyy-MM-dd'T'HH:mm:ss)

Revision: Leave blank to use the default version, which is managed by the integration.

Skip Related Object: Enable this to improve the performance.

Incremental: When running incremental mode, if start time and/or end time is missing, the connector uses the end time from previous run as new start time, and current system time as end time.

  1. Select Next.

Define Data Settings

ParameterTypeMandatoryDefault ValueDescription
Retry LimitNumberNo5Max retry count per API call. Min: 1, Max: 7
Initial retry time waitNumberNo1Initial retry interval per API call (second). Min: 1, Max: 10
Max retry waitNumberNo300Max retry interval per API call (second). Min: 30, Max: 1800
HTTP Connection TimeoutNumberNo300HTTP Connection Timeout (second). Min: 30, Max: 1800
HTTP read timeoutNumberNo300HTTP read timeout (second). Min: 30, Max: 1800
HTTP write timeoutNumberNo300HTTP write timeout (second). Min: 30, Max: 1800
  1. Select Next.

Preview Your Data

The data that displays in the data preview is approximated from your source. It is not the actual data that is imported.

  1. To preview your data, select Generate Preview. Optionally, select Next to skip to the next section.
  2. Verify that the data looks approximately like you expect it to.
  3. Select Next

Define Your Data Placement

In the Storage section specify details for where you want the data to reside in TD:

  • Database — Select the database where you want to save your data.
  • Table — The destination table where you want to store your imported data
  • Method
    • Append — Add records to your existing table. (Be aware that data may be duplicated.)
    • Always Replace — Always clear your destination table before adding records.
    • Replace on new data — If new data is found, then old data will be overwritten by the new data.
  • Timestamp-based Partition Key Select your custom timestamp column used as the partition key.
  • Data Storage Timezone — The expected timezone for your database.

In the Schedule section, you can choose when and how often you want to run this query.

  • Repeat — Select On or Off.
  • Schedule **—**The dropdown list  provides these options: @daily (midnight), @hourly,(:00), or  Custom cron.
  • Delay Transfer **—**You can specify a delay to the execution time.
  • Scheduling Timezone

Select Create & Run Now.

After your transfer has run, you can see the results of your transfer in Data Workbench > Databases

Import from Klaviyo via CLI (Toolbelt)

You import data from Klaviyo using the TD Toolbelt.

Prerequisites

Install the latest TD Tool using Ruby Gem.

$ gem install td
$ td --version
0.16.10

There are other install methods. For more information, see Treasure Data Toolbelt.

Create Config File

Create a config.yml configuration file:

in:
  type: klaviyo
  data_source: lists
  api_token: xxx
  revision_time: "2022-12-17"
  incremental: true
  start_time: "2023-06-01T17:06:39.698883"
  incremental_field: updated    
out:
  mode: append

Preview Data Coming In from Config File

You can preview the data your config file will pull in:

$ td connector:preview config.yml

+-----------+-----------------------+-------------------------+-------------------------+
| id:string |           name:string |          created:string |          updated:string |
+-----------+-----------------------+-------------------------+-------------------------+
|    SPg3aL |      chrome_line_name | 2021-08-30 05:35:46 UTC | 2021-08-30 05:35:46 UTC |
|    QPg3zL | 100_w_phone_line_name | 2021-08-27 23:49:56 UTC | 2021-08-27 23:49:56 UTC |
+-----------+-----------------------+-------------------------+-------------------------+

Scheduled Execution

You can schedule periodic data connector execution for periodic Klaviyo import. By using this feature, you no longer need a cron daemon on your local data center.

Create the Schedule

A new schedule can be created by using the td connector:create command. The following data needs to be specified:

  • Name of the schedule
  • The cron-style schedule
  • The database and table where their data will be stored
  • The Data Connector configuration file is required.
$ td connector:create \
  daily_klaviyo_import \
  "10 0 * * *" \
  sample_db \
  sample_table \
  config.yml

The cron parameter also accepts three special options: @hourly, @daily and @monthly. For more details, see  Scheduled Jobs.

By default, the schedule is set up in the UTC timezone. You can set the schedule in a timezone using the -tor --timezoneoption. The --timezoneoption supports only extended timezone formats like ‘Asia/Tokyo’, ‘America/Los_Angeles’ etc. Timezone abbreviations like PST and CST are not supported and may lead to unexpected schedules.

List the Schedules

You can see the list of currently scheduled entries by entering td connector:list.

$ td connector:list
  +----------------------+------------+----------+-------+-----------------+--------------+
  | Name                 | Cron       | Timezone | Delay | Database        | Table        |
  +----------------------+------------+----------+-------+-----------------+--------------+
  | daily_klaviyo_import | 10 0 * * * | UTC      | 0     | sample_database | sample_table |
  +----------------------+------------+----------+-------+-----------------+--------------+

Show the Setting and History of Schedules

td connector:show shows the execution setting of a schedule entry.

$ td connector:show daily_klaviyo_import  Name     : daily_klaviyo_import  Cron     : 10 0 * * *  Timezone : UTC  Delay    : 0  Database : sample_db  Table    : sample_table

td connector:historyshows the execution history of a schedule entry. To investigate the results of each individual execution, use td job:show jobid.

  | 577914 | success | 20000   | sample_db | sample_table | 0        | 2015-04-16 00:10:03 +0000 | 152      |  | 577872 | success | 20000   | sample_db | sample_table | 0        | 2015-04-15 00:10:04 +0000 | 163      |  | 577810 | success | 20000   | sample_db | sample_table | 0        | 2015-04-14 00:10:04 +0000 | 164      |  | 577766 | success | 20000   | sample_db | sample_table | 0        | 2015-04-13 00:10:04 +0000 | 155      |  | 577710 | success | 20000   | sample_db | sample_table | 0        | 2015-04-12 00:10:05 +0000 | 156      |  | 577610 | success | 20000   | sample_db | sample_table | 0        | 2015-04-11 00:10:04 +0000 | 157      |  +--------+---------+---------+-----------+--------------+----------+---------------------------+----------+

Delete the Schedule

td connector:deleteremoves the schedule.

$ td connector:delete daily_klaviyo_import

Appendix A: Klaviyo API Versioning

The TD Klaviyo connector only supports the "Stable" Klaviyo API. It does not support any "v1-2" legacy APIs nor any deprecated APIs. the "v1-2" legacy APIs are scheduled to end on January 1, 2024.

Klaviyo APIs follow a versioning system that allows for more explicit specification of precise API releases containing backwards-incompatible changes:

  • New Klaviyo APIs (/api endpoints) do not contain version numbers in their path.

  • Versions are now formatted with the pattern YYYY-MM-DD, e.g., revision 2022-04-15. In the Klaviyo connector, the revision is a required parameter when you create a new source using the TD Console or the TD Toolbelt.

  • Klaviyo recommends using the following revisions

    RevisionDeprecated?Planned retirement dateRetired?
    2022-10-17Yes~January 24th, 2024No
    2023-01-24Yes~February 23rd, 2024No
    2023-02-22NoTBDNo

References: