You can ingest data of AccountsProfile ManagementData Store, and Audit Log from Gigya (SAP Customer Data Cloud) into Treasure Data.

 Prerequisites

Incremental loading uses the maximum value (max value) in the specified incremental column to load all records until the max value for the first execution and subsequent runs imports records from (max value +1) from the previous run to the current time when the job runs, which becomes the new max value.

Supported



Limitations and Requirements

Query Syntax Limitation

Treasure Data supports the following SQL query syntax for Gigya:

Data Source Limitation

For Account Data Source

For Profile Management Data Source

For Data Store Data Source

For Audit Log Data Source


Incremental Loading and Numeric and Timestamp Columns

Incremental loading uses the maximum value (max value) in the specified incremental column to load all records till max value for the first execution and subsequent runs import records from (max value +1) from the previous run to the current time when the job runs (which becomes the new max value).

Support for:


Use the TD Console to Create Your Connection

Obtain your API Key, User Key, and Secret Key From Gigya

  1. Follow the instruction in Creating and Managing Applications to create your own application and obtain the App User Key and Secret Key.
  2. Follow the instructions in API Key and Site Setup to obtain your API Key.
  3. Follow the instructions to determine your Data Center.

Create a New Connection

When you configure a data connection, you provide authentication to access the integration. In Treasure Data, you configure the authentication and then specify the source information.

Open TD Console.
Navigate to Integrations Hub ->  Catalog
Search and select Gigya (SAP Customer Data Cloud).

The following dialog opens.

Choose your account Data Center.
Type values for the following:
    • API Key
    • User Key
    • User Secret

Select Continue.
Enter a name for your connection and select Done.

Transfer Your Gigya Accounts Data to Treasure Data

After creating the authenticated connection, you are automatically taken to the Authentications tab.

Search for the connection you created and select New Source.

Name the Source.
Select Next.
In the Source Table, edit the parameters.

Parameters

Description

Data Source

Target data source. Current support: Accounts, Profile Management, Data Store and Audit Log

Query

Gigya's query to ingest data. Depend on your target object, the query would be variant.

For Account and Profile Management data source, only support on accounts and emailAccounts object (sample query: Select * From accounts)

For Data Store it would be whatever your object is (sample query: Select * From my_data)

For Audit Log, only support for auditLog object (sample query: Select * From auditLog

Fields To Exclude

Due to Gigya's API specification, it is not possible to specify the columns to be included in the SELECT statement. This parameter can be used to remove unnecessary columns.

Batch Size

The maximum number of records to fetch in a single API call. The maximum value is 10000 and the minimum value is 10. When you customize the batch size, consider the following: a smaller value will let the API return faster but will cause more API calls.

Incremental

When running on a schedule, the next import ingests only the data that was updated after the last run based on the value of the Incremental Column.

Incremental Column

Which data object's column on which to perform the incremental transfer.

For Account and Profile Management data source, suggested values are: created, createdTimestamp, updated, and updatedTimestamp.

For Data Store data source, suggested values are: numeric or datetime colum

For Audit Log data source, suggested values are: @timestamp

Data Settings

In this dialog, you can edit data settings or opt to skip this step.


Data Preview 


Data Placement