Skip to content
Products
/
/
/
Attaching and Searching f...
Last updated

Attaching and Searching for Tags Using the REST API

In the following request and response examples, please note that the field sensitive is no longer supported in the UI for tagging columns. The sensitive field remains in the API only to support back compatibility for the legacy version of schema annotation.

You can annotate or search databases or tables based on tags, but you must have policy-based database permissions enabled. Learn more about access control for tags.

This article contains:

Attach Tags to a Column

You can attach a PII tag to a host column of a table. In the following example, the tag PII id with the tag id: ff0e21db-af92-419f-b936-0c9c1601f2f7 is being attached to the host column of a table using database_id:484834 and table_id: 2767455.

$ curl --location --request POST 'https://api-data-def-repo.treasuredata.com/v1/column-annotation/bulk?tableId=484834.2767455' \
    --header 'Authorization: TD1 123/abcdef•••••••••••••••••••••••0123456789' \
    --header 'Content-Type: application/json' \
    --data-raw '{
        "data": [
            {
                "attributes": {
                    "sensitive": false
                },
                "relationships": {
                    "annotation-type": {
                        "data": {
                            "id": "3213869e-7e13-45fc-99a7-1b039a7333a5",
                            "type": "annotation-type"
                        }
                    },
                    "column": {
                        "data": {
                            "id": "host",
                            "type": "treasure-data-column"
                        }
                    }
                },
                "type": "treasure-data-column"
            }
        ]
    }'

Response Example

{
  "data": [
    {
      "id": "ff0e21db-af92-419f-b936-0c9c1601f2f7",
      "attributes": {
        "comment": "Personal information",
        "createdAt": "2022-01-13T21:24:57.321823Z",
        "sensitive": false
      },
      "relationships": {
        "annotation-type": {
          "data": {
            "type": "annotation-type",
            "id": "3213869e-7e13-45fc-99a7-1b039a7333a5",
            "name": "PII"
          }
        },
        "table": {
          "data": {
            "type": "treasure-data-table",
            "id": "484834.2767455"
          }
        },
        "column": {
          "data": {
            "type": "treasure-data-column",
            "id": "host"
          }
        },
        "created-by": {
          "data": {
            "type": "treasure-data-user",
            "id": "18174"
          }
        }
      },
      "type": "column-annotation"
    }
  ],
  "links": {
    "first": "/v1/column-annotation?filter%5BtableId%5D=484834.2767455",
    "self": "/v1/column-annotation?filter%5BtableId%5D=484834.2767455"
  },
  "meta": {
    "perPage": 1
  }
}

Attach Tags to a Workflow

  1. Create a new resource tag using the POST method and the annotation-type parameter.
curl --location --request POST 'https://api-data-def-repo.treasuredata.com/v1/annotation-type/custom' \
--header 'Authorization: TD1 ${123/abcdef•••••••••••••••••••••••0123456789}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "data": {
      "attributes": {
        "color": "SECONDARY",
        "comment": "workflow-for-seasonal-sales",
        "name": "seasonal-sales",
        "humanReadableName": "seasonal-sales",
        "namespace": "RESOURCE"
        }
    }
}'

Example Response

You need to save the tag id to complete a later step. In the example response, the tag id is: ab48acf2-89f7-45a2-b7da-5569e47eb862.

{"id":"ab48acf2-89f7-45a2-b7da-5569e47eb862","attributes":{"name":"seasonal-sales","humanReadableName":"seasonal-sales","comment":"workflow-for-seasonal-sales","version":1,"createdAt":"2022-05-17T02:15:06.822978Z","updatedAt":"2022-05-17T02:15:06.822978Z","color":"SECONDARY","namespace":"RESOURCE"},"type":"annotation-type"}
  1. Get the project ID using the GET method and workflow's API /api/projects?name_pattern=blackfriday_sale to fetch the project id.
curl --location --request GET 'https://api-workflow.treasuredata.com/api/projects?name_pattern=blackfriday-sale' \
--header 'Authorization: TD1 ${123/abcdef•••••••••••••••••••••••0123456789}'

Example Response

Save the project id for later use. In the following example, the project id is: 187725.

{
    "projects": [
        {
            "id": "187725",
            "name": "blackfriday_sale",
            "revision": "26606e4c-93ec-456e-86b3-61390ce8ad1a",
            "createdAt": "2022-02-10T05:48:03Z",
            "updatedAt": "2022-02-10T05:48:03Z",
            "deletedAt": null,
            "archiveType": "s3",
            "archiveMd5": "Vw3N3vAtOMNq21mvQ2eRhg=="
        }
    ]
}
  1. Create tag for workflow

Use the POST method along with the query parameters filter[projectId] and filter[worfklowId] to attach tags to your workflows.

curl --location --request POST 'https://api-data-def-repo.treasuredata.com/v1/workflow-annotation/bulk?projectId=187725&workflowId=daily-ingestion' \
--header 'Authorization: TD1 ${123/abcdef•••••••••••••••••••••••0123456789}' \
--header 'Content-Type: application/json' \
--data-raw

Sample Payload Request

{
  "data": [
    {
      "type": "workflow-annotation",
      "attributes": {
        "comment": null
      },
      "relationships": {
        "annotation-type": {
          "data": {
            "id": "ab48acf2-89f7-45a2-b7da-5569e47eb862",
            "type": "annotation-type"
          }
        },
        "workflow": {
          "data": {
            "type": "treasure-data-workflow",
            "id": "daily-ingestions"
          }
        }
      }
    }
  ]
}

Response Example

{
    "data": [
        {
            "id": "ec6993ca-b14a-459e-8b44-2a63001799a1",
            "attributes": {
                "comment": null,
                "createdAt": "2022-05-17T05:28:57.029777Z"
            },
            "relationships": {
                "annotation-type": {
                    "data": {
                        "type": "annotation-type",
                        "id": "ab48acf2-89f7-45a2-b7da-5569e47eb862",
                        "name": "seasonal-sales"
                    }
                },
                "workflow": {
                    "data": {
                        "type": "treasure-data-workflow",
                        "id": "daily-ingestions"
                    }
                },
                "project": {
                    "data": {
                        "type": "treasure-data-workflow-project",
                        "id": "187725"
                    }
                },
                "created-by": {
                    "data": {
                        "type": "treasure-data-user",
                        "id": "1273"
                    }
                }
            },
            "type": "workflow-annotation"
        }
    ],
    "links": {
        "first": "/v1/workflow-annotation?filter%5BprojectId%5D=187725",
        "self": "/v1/workflow-annotation?filter%5BprojectId%5D=187725"
    },
    "meta": {
        "perPage": 1
    }
}

Search Databases Using Tags

You can search databases based on tags by using tags as filters. For instance, the following example shows using the PII tag and the result shows that the PII tag is found in the table with id 607222.3276426 and at column dob in the table.

$ curl -s --request GET "https://api-data-def-repo.treasuredata.com/v1/column-annotation/lookup?annotationTypeName=PII" \
--header 'Authorization: TD1 ${123/abcdef•••••••••••••••••••••••0123456789}'

Response Example

{
  "data": [
    {
      "id": "465302e6-67c0-4b33-9462-2f501c511601",
      "attributes": {
        "comment": "",
        "createdAt": "2021-09-30T07:40:01.792499Z",
        "sensitive": false
      },
      "relationships": {
        "annotation-type": {
          "data": {
            "type": "annotation-type",
            "id": "2f27b00e-9377-419e-91c3-4456598c30a8",
            "name": "PII"
          }
        },
        "table": {
          "data": {
            "type": "treasure-data-table",
            "id": "607222.3276426"
          }
        },
        "column": {
          "data": {
            "type": "treasure-data-column",
            "id": "dob"
          }
        },
        "created-by": {
          "data": {
            "type": "treasure-data-user",
            "id": "33648"
          }
        }
      },
      "type": "column-annotation"
    },
    ...
  ],
  "links": {
    "first": "/v1/column-annotation/lookup?annotationTypeName=PII",
    "self": "/v1/column-annotation/lookup?annotationTypeName=PII"
  },
  "meta": {
    "perPage": 250
  }
}

Search Projects Using Tags

You can use a GET command along with the query parameter annotationTypeName to search projects by tag. In the following example, the user is requesting to find a list of workflows that are tagged with daily_workflow.

curl --request GET 'https://api-data-def-repo.treasuredata.com/v1/project-annotation?annotationTypeName=daily-workflow' \ \
--header 'Authorization: TD1 ${123/abcdef•••••••••••••••••••••••0123456789}'

Sample Response

The response means there is only 1 workflow that is tagged with daily_workflow. The workflow is called workflow_name (in project 187725).

{
        "data": [
            {
                "id": "4ad2008c-1019-4ced-8e53-74b155df0c0f",
                "attributes": {
                    "comment": null,
                    "createdAt": "2022-05-17T02:50:36.586640Z"
                },
                "relationships": {
                    "annotation-type": {
                        "data": {
                            "type": "annotation-type",
                            "id": "4a45b1de-b27b-4414-92d6-4224490a5078",
                            "name": "daily-workflow"
                        }
                    },
                    "project": {
                        "data": {
                            "type": "treasure-data-workflow-project",
                            "id": "187725"
                        }
                    },
                    "created-by": {
                        "data": {
                            "type": "treasure-data-user",
                            "id": "1273"
                        }
                    }
                },
                "type": "project-annotation"
            }
        ],
        "links": {
            "first": "/v1/project-annotation?annotationTypeName=daily-workflow",
            "self": "/v1/project-annotation?annotationTypeName=daily-workflow"
        },
        "meta": {
            "perPage": 250
        }
    }

Search Workflows Using Tags

You can use a GET command along with the query parameter annotationTypeName to search workflows by tag. In the following example, the user is requesting to find a list of workflows that are tagged with the tag daily_workflow.

curl --location --request GET 'https://api-data-def-repo.treasuredata.com/v1/workflow-annotation?annotationTypeName=daily-workflow' \ \
--header 'Authorization: TD1 ${123/abcdef•••••••••••••••••••••••0123456789}'

Sample Response

The response means there is only 1 workflow that is tagged with daily-workflow. The workflow is called daily-ingestions (in project 187725).

{
        "data": [
            {
                "id": "ec6993ca-b14a-459e-8b44-2a63001799a1",
                "attributes": {
                    "comment": null,
                    "createdAt": "2022-05-17T05:28:57.029777Z"
                },
                "relationships": {
                    "annotation-type": {
                        "data": {
                            "type": "annotation-type",
                            "id": "4a45b1de-b27b-4414-92d6-4224490a5078",
                            "name": "daily-workflow"
                        }
                    },
                    "workflow": {
                        "data": {
                            "type": "treasure-data-workflow",
                            "id": "daily-ingestions"
                        }
                    },
                    "project": {
                        "data": {
                            "type": "treasure-data-workflow-project",
                            "id": "187725"
                        }
                    },
                    "created-by": {
                        "data": {
                            "type": "treasure-data-user",
                            "id": "1273"
                        }
                    }
                },
                "type": "workflow-annotation"
            }
        ],
        "links": {
            "first": "/v1/workflow-annotation?annotationTypeName=daily_workflow",
            "self": "/v1/workflow-annotation?annotationTypeName=daily_workflow"
        },
        "meta": {
            "perPage": 250
        }
    }
Previous page
Next page