The Docker images provided by Treasure Data for Custom Scripts have versions like “3.9.1-td2”. This version is formatted as follows:
PYTHON_VERSION-SECURITY_FIX
“3.9.1” is PYTHON_VERSION. It is a version of Python included in the image.
“-td2” is SECURITY_FIX. It is a sequential number which is incremented when Treasure Data releases a security hotfix. This number begins with “td0”.
Therefore, if the Docker image version is “3.9.1-td2”, it means that the included Python version is 3.9.1 and this is the second security fix release for the 3.9.1 series.
In this scheme, the Docker image used in your Custom Script task will never be upgraded without your operation. When a new Docker image is released, you need to test your Custom Script task with the new version and upgrade to it.
Treasure Data will release new Docker images for Custom Scripts when:
A new version of Python is released.
Any security fix is needed.
Details follow.
Treasure Data will provide a new Docker image for Custom Scripts when a new version of Python is released. For example, when Python 3.77.0 is released, we will release the Docker image “3.77.0-td0”.
Note that our release times are best-effort. The interval between the release of a new version of Python and the release of a Docker image for Custom Scripts is not guaranteed. Furthermore, Docker images for Custom Script will not be released for all versions of Python.
Treasure Data will provide a new Docker image for Custom Scripts if we identify a critical security vulnerability that needs to be fixed in any of the components contained in the released Docker image, and the security fix is already provided by the upstream, and we accept that the fix is needed. For example, when a security fix is provided for the critical security vulnerability for some library included in the “3.77.2-td0” image and we accept it is needed to be fixed, we will release a new Docker image “3.77.2-td1” with the security fix.
Treasure Data will provide security fix releases for the Docker image versions that have two latest Python minor versions. For example, the latest Python minor versions of our latest Docker images are 3.10 and 3.9 as of now, so we will provide security fixes for 3.10 and 3.9 series images.
Regarding regular releases (non-security fix releases), backward compatibility is not guaranteed. For example, Docker image 3.12.2-td0 could be backward incompatible with 3.12.1-td0. You need to test your Custom Script task with the target image.
Regarding security fix releases, Treasure Data will try to keep backward compatibility as much as we can. However, 100% backward compatibility is not guaranteed for any release, depending on the upstream change or dependent libraries. For example, Docker image 3.12.1-td1 is expected to be mostly backward compatible with 3.12.1-td0, but not guaranteed.
If two newer Python minor versions are released than the version of a given Docker image, the image becomes obsolete and no security fix is provided. You can still use this version for a while after that, but after a certain amount of time it will be removed after notification.
For example, Docker image “3.10.9-td0” will be obsolete after “3.12.0-td0” and “3.11.0-td0” is released.
Treasure Data previously provided Docker images with a different versioning scheme to that described above. Those images had Docker image versions like “3.9” or “3.9.1”. Those versions continue to work for a while, but these versions are obsolete and not recommended for use in workflows that are now being written. Those versions will be removed following the deprecation policy described above, too.