This data connector is in Beta. For more information, contact support@treasuredata.com.

Commerce Cloud connects the customer journey and drives success from anywhere with customer-centric commerce built for every industry.

  • Completely connect your digital customer journey.
    • Personalize engagement with customer-centric commerce.
    • Unify data, personalize every interaction, and grow revenue across channels with automation, AI, and a single source of truth.
  • Connect commerce to the customer journey.

    • Convert more customers and drive loyalty with a seamless customer journey from marketing to sales, commerce, fulfillment, service, and beyond. Build with your choice of clicks- or code-based tools.
  • Adapt quickly with an agile, scalable, and secure platform.
    • Innovate at the speed of your customers, scale easily across the globe, and meet any level of demand. Extend your commerce with a partner ecosystem of apps to deliver experiences like augmented reality, marketplaces, and more.

What can you do with this Integration?

  • Salesforce Commerce Cloud Import Integration will provide the following abilities:

    • Authenticate using API Client via OAuth.

    • Ingest customer information for a specific list

    • Ingest of Customer resources, for example:

      • customer information

      • name, gender, email

    • Ingest Product resource, for example:

      • a list of products

      • product information

    • Ingest Inventory and Product Inventories resource, for example:
      • list of inventories
      • product inventory allocation
      • product inventory stock level 
      • product inventory ID
    • Ingest Category and Product Assignments resource, for example:
      • category information
      • product belongs to each category
    • Ingest Store resources, for example:

      • all stores

      • store information including id, address, postal code, country, inventory id

    • Ingest Catalog resources, for example:
      • catalog info
      • assigned product counts



Prerequisites

  • Basic Knowledge of Treasure Data

  • Basic knowledge of Treasure Data CLI

  • Basic knowledge of Salesforce Commerce Cloud
  • Basic knowledge of Open Commerce Cloud API
  • Commerce Cloud Account Manager System Access

Limitations

  • Incremental loading is only supported for Creation Date, not Modified Date
  • A large volume of data may cause low performance
  • In preview mode, the connector does not support generating the detailed data preview because of time restrictions on fetching the data preview.

An Inventory List with no product is still inserted as one record, the properties belong to the Product Inventory List and will be null.

About Incremental Data Loading  


Incremental loading is the activity of loading only new or updated records from a source into Treasure Data. Incremental loads are useful because they run efficiently when compared to full loads, and particularly for large data sets.

Incremental loading is available for many of the Treasure Data integrations. In some cases, it is a simple checkbox choice and in others, after you select incremental loading you are provided with other fields that must be specified. 

Limitations, Supported, Suggestions

  • For some integrations, if you choose incremental loading, you might need to make sure that there is an index on the columns to avoid a full table scan.
  • Only Timestamp, Datetime, and numerical columns are supported as incremental_columns.
  • For the raw query, the incremental_columns is required because it won't be able to detect the Primary keys for a complex query.

About Incremental Loading for Integrations

Treasure Data Incremental loading has 4 patterns (3 types of data connector + 1 workflow td_load operator.), then the 3 data connector loading examples are as follows:

  • Cloud storage service (e.g. AWS S3, GCS and etc.)

    • Lexicographic order of file name

  • Query (e.g. MySQL, BigQuery and etc.)

    • Date time

  • Variable period (Google Analytics, etc)

    • Use start_date for loading

Incremental Loading for Connectors

If incremental loading is selected, data for the connector is loaded incrementally.

This mode is useful when you want to fetch just the object targets that have changed since the previously scheduled run.

For example, in the UI:

Database integrations, such as MySQL, BigQuery, and SQL server, require column or field names to load incremental data. For example:

Learn more About Database-based Integrations.


Create the API Client on Commerce Cloud Account Manager System

These steps are necessary to create the credentials that are needed to define the authentication for your integration.


1. Navigate to the Salesforce Cloud Commerce UI and login. For example, Account Manager UI.


2. Select API Client.

3. Select Add API Client.

4. Type values for:
    • Display Name
    • Password
    • Confirm Password
5. Select Access Control to enable it.
6. In the Organizations area, select your organization.
7. In the Roles area, select Add.
8. Select the appropriate role. For example, Sandbox API User.

9. Select Add.
10. After the role is added, select all the sandboxes associated with it.

11. Select Add.
12. Scroll down to Open ID Connect.
13. Add Default Scopes:
    • mail
    • roles
    • tenantFilter
    • profile
14. Add Redirect URI. For example:

https://admin.us01.dx.commercecloud.salesforce.com/oauth2-redirect.html

15. Scroll to the Token Endpoint Auth Method section.
16. Select client_secret_basic.
17. Select JWT for Access Token Format.

18. Select Save.

You are taken back to the API Client screen.



19. Write down or save in a secure location the API Client ID and Password.

These are used as Authentication credentials to integrate with your SFCC instance.


Enable the API Client to Access the Data API

These steps enable your API Client to be used with the Data API and Salesforce B2C Commerce Cloud connector.

1. Open Business Manager. For example:

https://{YOUR_INSTANCE_URL}.commercecloud.salesforce.com/on/demandware.store/Sites-Site/default/ViewApplication-DisplayWelcomePage

2. Select Administration.
3. Select Open Commerce API Settings.


4. Select Type as Data.

The configuration for Data API appears.

5. Type or copy the following into the text field:
{ "_v": "21.6", "clients": [ { "client_id": "your_api_client_id", "resources": [ { "methods": [ "get", "post", "put", "patch", "delete" ], "read_attributes": "(**)", "write_attributes": "(**)", "resource_id": "/**" } ] } ] }
6. For Select Context choose Global (organization-wide).

7. Select Save.
8. Wait for at least 3 minutes.

Your API Client is ready to be used with Data API and Salesforce B2C Commerce Cloud connector.

Use the TD Console to Create Your Connection

Create a New Connection

In Treasure Data, you must create and configure the data connection prior to running your query. As part of the data connection, you provide authentication to access the integration.

1. Open TD Console.
2. Navigate to Integrations Hub >  Catalog.
3. Search for and select Salesforce B2C Commerce Cloud.

4. Choose Create Authentication.

The following dialog opens:

5. Type or select the required information:
ParameterDescription

API Client ID

Key which you got after create your application on Commerce Cloud's Account Manager Application.

API Client Password

The password for your API Client when you create on Commerce Cloud's Account Manager Application.

Base URI

URL for accessing your Business Manager Application. Example value https://xxxx.commercecloud.salesforce.com/

6. Type a name for your connection.

7. Select Continue.



Transfer Your Data to Treasure Data


After creating the authenticated connection, you are automatically taken to Authentications.


1. Search for the connection you created. 

2. Select New Source.
3. Type a name for your Source in the Data Transfer field.

4. Select Next.

The Source Table dialog opens.

5. Edit the following parameters:

Parameters DescriptionDescription of Data Types
Data TypeThe data objects that you want to ingest into Treasure Data:

Customers From List

The customer info from a specific customer list


Products

Product info of Salesforce Commerce Cloud account


Categories and Product Assignments

Product category info and products belong to each category


Inventory Lists and Product Inventories

Inventory list and product in each inventory list


StoresStore info, location, and inventory belongs to stores

CatalogsProduct catalog info and count of products for each catalog 
Customer List IDThe customer List ID you want to import customer information from
Incremental loadingSelect to ingest only newly created data since the previous ingestion.
Created FromThe specific time you would like to ingest the data (YYYYMMDDHH format).
6. Select Next.

The Data Settings page can be modified for your needs or you can skip the page.

7. Optionally, edit the following parameters:

8. Select Next.

Data Preview 


You can see a preview of your data before running the import by selecting Generate Preview.

Data shown in the data preview is approximated from your source. It is not the actual data that is imported.

  1. Select Next.
    Data preview is optional and you can safely skip to the next page of the dialog if you want.

  2. To preview your data, select Generate Preview. Optionally, select Next

  3. Verify that the data looks approximately like you expect it to.


  4. Select Next.


Data Placement

For data placement, select the target database and table where you want your data placed and indicate how often the import should run.

  1.  Select Next. Under Storage you will create a new or select an existing database and create a new or select an existing table for where you want to place the imported data.

  2. Select a Database > Select an existing or Create New Database.

  3. Optionally, type a database name.

  4. Select a Table> Select an existing or Create New Table.

  5. Optionally, type a table name.

  6. Choose the method for importing the data.

    • Append (default)-Data import results are appended to the table.
      If the table does not exist, it will be created.

    • Always Replace-Replaces the entire content of an existing table with the result output of the query. If the table does not exist, a new table is created. 

    • Replace on New Data-Only replace the entire content of an existing table with the result output when there is new data.

  7. Select the Timestamp-based Partition Key column.
    If you want to set a different partition key seed than the default key, you can specify the long or timestamp column as the partitioning time. As a default time column, it uses upload_time with the add_time filter.

  8. Select the Timezone for your data storage.

  9. Under Schedule, you can choose when and how often you want to run this query.

    • Run once:
      1. Select Off.

      2. Select Scheduling Timezone.

      3. Select Create & Run Now.

    • Repeat the query:

      1. Select On.

      2. Select the Schedule. The UI provides these four options: @hourly, @daily and @monthly or custom cron.

      3. You can also select Delay Transfer and add a delay of execution time.

      4. Select Scheduling Timezone.

      5. Select Create & Run Now.

 After your transfer has run, you can see the results of your transfer in Data Workbench > Databases.

Available Configuration

Configuration name

Type

Available when

Required

Sample value

Description

base_url

STRING

default

yes

https://xxxx.example.com

SFCC’s instance base url

catalog_id

STRING

target = categories

no

test

Catalog’s ID

client_id

STRING

default

yes

abcdef

Application’s ID

client_secret

STRING

default

yes

abcdef

Application’s Secret

customer_list_id

STRING

target = customers

yes if target = customers

test

Customer List Id

incremental

BOOLEAN

target = customers

target = products

target = catalog

no

true

Ingest incremental or not

initial_retry_wait

LONG

default

no

1

First wait time when retrying (in seconds)

inventory_list_id

STRING

target = inventory_lists

no

test

Inventory List’s ID

last_value

STRING

target = customers

target = products

target = catalog

no

2021-05-25T02:49:29.000Z

The timestamp to start ingest data

levels

LONG

target = categories

yes if target = categories

0

Level of sub-category to ingest when target = categories

maximum_retries

LONG

default

no

5

Maximum number of retry

maximum_retry_wait

LONG

default

no

120

Max wait time when retrying (in seconds)

site_id

STRING

target = products

target = stores

yes if target = stores

test

The Site ID

target

ENUM (customers , stores , categories , products , catalogs , inventory_lists )

default

yes

customers

Target data type to ingest

Sample Configuration

Customers

in:
  type: salesforce_commerce_cloud
  client_id: xxx
  client_secret: yyy
  base_url: https://{realm_id}-001.sandbox.us01.dx.commercecloud.salesforce.com/
  target: customers
  customer_list_id: phu_test
  last_value: 2021-05-25T02:49:29.000Z
  incremental: true

Products

in:
  type: salesforce_commerce_cloud
  client_id: xxx
  client_secret: yyy
  base_url: https://{realm_id}-001.sandbox.us01.dx.commercecloud.salesforce.com/
  target: products
  site_id: phu_test
  incremental: true
  last_value: 2021-05-25T02:55:28.000Z

Stores

in:
  type: salesforce_commerce_cloud
  client_id: xxx
  client_secret: yyy
  base_url: https://{realm_id}-001.sandbox.us01.dx.commercecloud.salesforce.com/
  target: stores
  site_id: phu_test2

Inventory Lists and Product Assignments

in:
  type: salesforce_commerce_cloud
  client_id: xxx
  client_secret: yyy
  base_url: https://{realm_id}-001.sandbox.us01.dx.commercecloud.salesforce.com/
  target: INVENTORY_LISTS

Categories

in:
  type: salesforce_commerce_cloud
  client_id: xxx
  client_secret: yyy
  base_url: https://{realm_id}-001.sandbox.us01.dx.commercecloud.salesforce.com/
  target: categories
  catalog_id: phu_cat

Catalogs

in:
  type: salesforce_commerce_cloud
  client_id: xxx
  client_secret: yyy
  base_url: https://{realm_id}-001.sandbox.us01.dx.commercecloud.salesforce.com/
  target: catalogs
  incremental: true
  last_value: 2021-05-26T02:55:28.000Z


  • No labels