Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Continue to the following topics:

Table of Contents

Limitations

  • Unsupported SOQL: This configuration allows to use a custom SOQL (Salesforce Object Query Language) syntaxto query and filter data. With SOQL, you can retrieve the data from a single object or multiple objects that are related to each other. You can pick specific columns and filter or sort the output with your own conditional statement. As a limitation, our data connector doesn’t support SOQL syntax such as count() FROM Object and SELECT * from salesforceObject.
  • Bulk API limits. Bulk : Bulk API offers faster ingestion but also has a limitation of 10,000 batch allocations within a 24-hour period. If your target is large, your entire available batch allocation might be consumed, causing your job to fail eventually. If you try to use the Bulk API, and the result is the ingestion of all records, consider enabling the synchronous transfer only option and using REST API. The use of REST API avoids the batch allocation limitation but might be slower.

...

SFDC import supports these authentication types:

  • Credential

  • OAuth

Using Session ID for authentication is not supported for import.

Session ID is supported for the Salesforce Export Integration only.

Connect to Salesforce Using TD Console

Connecting to Salesforce using the TD Console is quick and easy. Alternatively, create Salesforce connection using the command line. The import integration supports credentials; you need a client ID and client secret to authenticate using credentials.

Allow TD to Access Salesforce using Salesforce

Excerpt

These instructions guide you to locate the client ID and client secret that is necessary to authenticate using credentials.

  1. Go to Setup > Apps > App Manage.

  2. Select New Connected App.


    The steps may vary depending on the version of Salesforce you are running. In Spring 19 classic UI: Setup > Build > Create > Apps > Connected Apps > New.

    Image Modified


  3. Navigate to Setup > Build > Create (Apps), and validate all your connected apps:

    Image Modified


  4. Select your app name to go to the page where you can view and manage all information about your connected app.

  5. Write down or copy your Consumer Key (client_id) and Consumer Secret (client_secret).

    Image Modified


  6. For secure account access, get a Salesforce Security Token. If you don’t have a security token, go to Account > Settings > Reset My Security Token and select Reset Security Token. You’ll receive your Security Token by email.

    In Spring 19 classic UI: My account > My Settings > Personal > Reset My Security Token.

Create New Connection in TD Console

...

Expand
titleTo authenticate with Credentials
  1. To authenticate with your credentials, enter your username (your email) and password, as well as your Client ID, Client Secret, and Security Token.

  2. In the dialog box, enter login.salesforce.com as the login URL. If you are using newer version Salesforce v2 connector, make sure to remove  Remove unnecessary letters from Login URL parameter. For example, instead of login.salesforce.com/?locale=jp, use login.salesforce.com.

Image Modified



Expand
titleTo authenticate with OAuth
Image Modified


5. Select Continue.

...

To validate the user permission, make sure to use Salesforce to validate:

  • Authority: check the Salesforce import integration connection steps.

  • Allowed access to Salesforce from Treasure Data: sometimes configuring this requires knowing and configuring your TD static IP address. Please contact support if you require the static IP address information.

Without properly configured authority and access, you might encounter access restriction errors. For example:

Code Block
linenumberstrue
Response not 2xx: 
400 Bad Request {"error":"invalid_grant","error_description":"authentication failure"}

...

  1. Search for the connection you created. 

  2. Select New Source.

  3. Type a name for your Source in the Data Transfer field.

    Image Modified


  4. Click Next

  5. Edit the following parameters:

    Image Modified


Parameters

Description

Source

Name of the object you want to import

Include deleted records

Enables including deleted records

Use synchronous transfer only

Enables synchronous transfer using REST API

Incremental

Imports only new data since the last import

Data Settings

  1. Select Next. The Data Settings page opens.

  2. You can edit the SOQL query, WHERE conditions and Schema settings here.

  3. Optionally, skip this page of the dialog.

    Image Modified


Data Preview 

...