Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Learn more about Import Integration Filters:

Table of Contents
maxLevel1

See also About Filter Functions .

...

These filters can be used during import to modify your data. They can be used to retain columns, add columns, drop columns, expand JSON columns, or mask column data. Masking data is typically used to secure sensitive information such as social security numbers. You interact with the filters as part of defining your TD import integrations.

Image Modified


Image Modified

Types of Filters

Filter Type

Description

Use Case Example

Retain Columns

Use to specify the columns that you would like to keep.

Your sales team wants a table with all of your users' names and phone numbers. You can import a data set of user registrations from your website, but that data set has names, phone numbers and 50 other columns that you don’t need. The retain filter allows you to specify that only the name and phone number columns are ingested.

Add Columns

Use to duplicate a column or even create a new column with a static value. It can also copy an existing column and rename it to a new value.

You want to add a column named ‘source’ with a value of “12345” to all rows that come from the job you’ve setup for your Facebook Ad with the ID of 12345. The Add Columns Filter can do this. It can also copy an existing column and rename it to a new value.

Drop Columns

Use to drop a column from data you are importing.

Your marketing team wants a list of customer information such as name, email, address, phone, and certain preferences and that data exists in a table which also contains some sensitive information or values that are not needed, you can drop those columns.

Expand JSON

Turns JSON objects in your data set into multiple new columns.

If you had a column with a JSON object like: {“first_name”: “John”, “last_name”: “Doe”}, the Expand JSON Filter could extract those values from your JSON and create the following new columns:

  • first_name

  • last_name

Digest

Use to protect sensitive information from being exposed. The data you are importing may contain a column that you deem sensitive or private. The digest filter allows you to hash the data in that specific column. You can choose the algorithm that is used to hash each column.

Your marketing team wants a segment list of customer contact information that also includes social security numbers. You can use the digest filter to hide that information.

Learn more about applying import integration filters:

Children Display

Order of Operations

Filters run after decoders and parsers. In the TD Console, your data is initially be decoded, parsed and then filtered. Although your source data may have certain schemas or names, the filters only see what comes out of the parser.

...

This happens because the Retain Columns Filter already removed the car_vin column and the Digest Filter only sees what came out of the filter before it as follows:

Image Modified