You can connect Salesforce with Treasure Data for greater control over your Salesforce data and better integration with the other business applications in your marketing and sales operations stack.
Integrating Salesforce with Treasure Data makes it easy to:
Add new features to Salesforce. For example, you can prevent churn by tracking web usage and receiving alerts when customers’ product usage declines.
Use Salesforce data to improve other parts of your marketing stack. For example, you can increase your Facebook Ads ROI by automatically removing new customers from your Facebook Custom Audiences.
For sample workflows on importing data from Salesforce, go to Treasure Boxes.
If you don’t have a Treasure Data account yet, contact us so that we can get you set up.
Continue to the following topics:
Limitations
Unsupported SOQL (Salesforce Object Query Language) syntax. As a limitation, our data connector doesn’t support SOQL syntax such as count() FROM Object.
Bulk API limits. Bulk API offers faster ingestion but also has a limitation of 10,000 batch allocations within a 24-hour period. If your target is large, your entire available batch allocation might be consumed, causing your job to fail eventually. If you try to use the Bulk API, and the result is the ingestion of all records, consider enabling the synchronous transfer only option and using REST API. The use of REST API avoids the batch allocation limitation but might be slower.
Using Session ID for authentication is not supported for import. The session ID is supported for the Salesforce Export Integration only.
Unsupported OAuth. Salesforce Import Integration does not support OAuth for non-standard login URLs (other than https://logins.salesforce.com/).
Support
SFDC import supports these authentication types:
Credential
OAuth
Connect to Salesforce Using TD Console
Connecting to Salesforce using the TD Console is quick and easy. Alternatively, create Salesforce connection using the command line. The import integration supports credentials; you need a client ID and client secret to authenticate using credentials.
Allow TD to Access Salesforce using Salesforce
These instructions guide you to locate the client ID and client secret that is necessary to authenticate using credentials.
Go to Setup > Apps > App Manage.
Select New Connected App.
The steps may vary depending on the version of Salesforce you are running. In Spring 19 classic UI: Setup > Build > Create > Apps > Connected Apps > New.
Navigate to Setup > Build > Create (Apps), and validate all your connected apps:
Select your app name to go to the page where you can view and manage all information about your connected app.
Write down or copy your Consumer Key (client_id) and Consumer Secret (client_secret).
For secure account access, get a Salesforce Security Token. If you don’t have a security token, go to Account > Settings > Reset My Security Token and select Reset Security Token. You’ll receive your Security Token by email.
In Spring 19 classic UI: My account > My Settings > Personal > Reset My Security Token.
Create New Connection in TD Console
Open TD Console.
Navigate to Integrations Hub > Catalog and search for Salesforce.
To authenticate with your credentials, enter your username (your email) and password, your Client ID, Client Secret, and Security Token.
In the dialog box, enter login.salesforce.com/?locale=jp, or use login.salesforce.com.
5. Select Continue.
6. Give your connection a descriptive name and select Create Connection.
Validate Your Salesforce Connection
To validate the user permission, make sure to use Salesforce to validate:
Authority: check the Salesforce import integration connection steps.
Allowed access to Salesforce from Treasure Data: sometimes configuring this requires knowing and configuring your TD static IP address. Contact support if you require the static IP address information.
Without properly configured authority and access, you might encounter access restriction errors. For example:
Response not 2xx:
400 Bad Request {"error":"invalid_grant","error_description":"authentication failure"}
Transfer Your Salesforce Account Data in Treasure Data
After creating the authenticated connection, you are automatically taken to Authentications.
Search for the connection you created.
Select New Source.
Type a name for your Source in the Data Transfer field.
Click Next.
Edit the following parameters:
Parameters | Description |
Source | Name of the object you want to import |
Include deleted records | Enables including deleted records |
Use synchronous transfer only | Enables synchronous transfer using REST API |
Incremental | Imports only new data since the last import |
Data Settings
Select Next. The Data Settings page opens.
You can edit the SOQL query, WHERE conditions and Schema settings here.
Optionally, skip this page of the dialog.
Data Preview
You can see a preview of your data before running the import by selecting Generate Preview. Data shown in the data preview is approximated from your source. It is not the actual data that is imported. Click Next. To preview your data, select Generate Preview. Optionally, click Next. Verify that the data looks approximately like you expect it to. Select Next.
Data preview is optional and you can safely skip to the next page of the dialog if you want.
Data Placement
For data placement, select the target database and table where you want your data placed and indicate how often the import should run. Select Next. Under Storage you will create a new or select an existing database and create a new or select an existing table for where you want to place the imported data. Select a Database > Select an existing or Create New Database. Optionally, type a database name. Select a Table> Select an existing or Create New Table. Optionally, type a table name. Choose the method for importing the data. Append (default)-Data import results are appended to the table. Always Replace-Replaces the entire content of an existing table with the result output of the query. If the table does not exist, a new table is created. Replace on New Data-Only replace the entire content of an existing table with the result output when there is new data. Select the Timestamp-based Partition Key column. Select the Timezone for your data storage. Under Schedule, you can choose when and how often you want to run this query. Select Off. Select Scheduling Timezone. Select Create & Run Now. Repeat the query: Select On. Select the Schedule. The UI provides these four options: @hourly, @daily and @monthly or custom cron. You can also select Delay Transfer and add a delay of execution time. Select Scheduling Timezone. Select Create & Run Now. After your transfer has run, you can see the results of your transfer in Data Workbench > Databases.
If the table does not exist, it will be created.
If you want to set a different partition key seed than the default key, you can specify the long or timestamp column as the partitioning time. As a default time column, it uses upload_time with the add_time filter.