Treasure Data allows the import of your data from Jira.
Basic knowledge of Treasure Data.
Deprecation notice from Jira
Atlassian Corporation announced changes to Jira. Effective December 2018, the basic authentication method that uses passwords and cookie-based authentication to access their APIs, is deprecated. Update existing authentication as quickly as possible. Use the email and API key method when creating your authentication.
How to obtain Jira API key from your Jira web application
Login to your Jira web application server. The following example uses Jira cloud (https://id.atlassian.com)
Go to https://id.atlassian.com/manage-profile/email and note your email address. Users with basic authentication should replace the user name value with the email address.
Select Security then select Create or manage API tokens (or you can go directly to the link (https://id.atlassian.com/manage/api-tokens).
Next, select Create API token.
In the pane that appears, enter a name for your API Token and select Create.
Then on the next pane, select View to see your new API keys.
You can now see your API key. Copy it and use it to authenticate the TD Console to your Jira server.
How to transfer your Jira data to Treasure Data using the TD Console
Go to Integration Hub -> Catalog and select the JIRA tile.
Enter Email, API Key and JIRA API endpoint
Select Continue and enter a name for your new connection.
Go back to Integrations Hub > Authentications then select your created authentication and select New Source.
Enter a valid JQL in the New Source pop up then select Next.
When you see the preview, you can select Advanced Settings if you want to remove, add, or modify the columns which the plugin will attempt to ingest. Select Next.
In the Transfer to tab, enter the database name and table name that you want to insert the data into.
If you are creating a new database, select Create new database and give your database a name. Complete the same steps for Create new table.
Select whether to append records to an existing table or replace your existing table.
If you want to set a different partition key seed rather than use the default key, you can specify one using the popup menu.
In the Schedule tab, you can specify a one-time transfer, or you can schedule an automated recurring transfer.
Select Next. You can, optionally, rename the source. Select Done and your job is scheduled. You will be notified whenever it is done.
How to transfer your Jira data to Treasure Data using TD Toolbelt
Install ‘td’ command v0.11.9 or later
Install the newest TD Toolbelt.
Create Seed Config File (seed.yml)
seed.yml as shown in the following example, with your JIRA email with API key, JIRA URI and, by default, an empty jql query. (Username and password is not longer supported for authentication). You’ll use “replace” mode:
The Data Connector for Jira imports all tickets that match the specified JQL.
For more details on available
out modes, see Appendix.
Guess Fields (Generate load.yml)
connector:guess. This command automatically reads the target data, and intelligently guesses the data format.
If you open up
load.yml, you’ll see guessed file format definitions including, in some cases, file formats, encodings, column names, and types.
Then you can preview how the system will parse the file by using
If the system detects your column name or type unexpectedly, modify
load.yml directly and preview again.
Currently, the Data Connector supports parsing of "boolean", "long", "double", "string", and "timestamp" types.
Execute Load Job
Submit the load job. It may take a couple of hours depending on the data size. Users need to specify the database and table where their data is stored.
The preceding command assumes you have already created database(td_sample_db) and table(td_sample_table). If the database or the table do not exist in TD, this command will not succeed, so create the database and table manually or use
--auto-create-table option with
td connector:issue command to auto create the database and table:
You can assign Time Format column to the "Partitioning Key" by "--time-column" option.
You can schedule a periodic Data Connector execution for periodic JIRA import. We manage our scheduler carefully to ensure high availability. By using this feature, you no longer need a
cron daemon on your local data center.
For the scheduled import, Data Connector for JIRA imports all files that match the specified JQL.
Create the schedule
A new schedule can be created by using the
td connector:create command. The name of the schedule, cron-style schedule, the database and table where their data will be stored, and the Data Connector configuration file are required.
The `cron` parameter also accepts three special options: `@hourly`, `@daily` and `@monthly`.
By default, schedule is setup in UTC timezone. You can set the schedule in a timezone using -t or --timezone option. The `--timezone` option supports only extended timezone formats like 'Asia/Tokyo', 'America/Los_Angeles' etc. Timezone abbreviations like PST, CST are *not* supported and can lead to unexpected schedules.
List the Schedules
You can see the list of currently scheduled entries by
Show the Setting and History of Schedules
td connector:show shows the execution setting of a schedule entry.
td connector:history shows the execution history of a schedule entry. To investigate the results of each individual execution, use
td job <jobid>.
Delete the Schedule
td connector:delete removes the schedule.
Modes for out plugin
You can specify file import mode in
out section of seed.yml.
This is the default mode and records are appended to the target table.
replace (In td 0.11.10 and later)
This mode replaces data in the target table. Any manual schema changes made to the target table remain intact with this mode.