The hash filter plugin for Treasure Data's Data Connectors allows you to specify the hashing algorithm on certain columns.


Options

This plugin contains the following option:

Option

Description

column_options

a key-value pairs. Key is a column name and Value is for column options.

Child Elements

Description

algorithm

The algorithm to digest. Can be: MD5, SHA-1, or SHA-256 (string, required)

salt

The salt value. When present, "salt value + original string" is digested (string, optional)

salt_column

The column that stores the salt value. When present, "salt column's value + original string" is digested (string, optional)

rename_to

Used to change the column name (string, optional)

charset_to_encode_strings

The character set encoding name. Used to encode strings in the column that your Java Runtime supports (string, default: UTF-8).


Configuration

Add the filter section, including the type, to your load.yml. In the following example, the type is digest:

in:
...
filters:
- type: digest
  column_options:
    id:
      algorithm: MD5
    name:
      algorithm: SHA-1
      rename_to: name_digest
    email:
      algorithm: SHA-256
      salt: salt_value
    phone:
      algorithm: SHA-256
      salt_column: col1
out:
...
  • No labels