FAQ for Data Connector

Here are some DataConnector questions frequently asked by our users.

Table of Contents

Q: Data Connector for S3 job is running for a long time, what to do next?

  • Please check the count of S3 files that your connector job is ingesting. If there are over 10,000 files, the performance would get worse. To mitigate this issue, we could:
    • Narrow path_prefix option and reduce the count of S3 files.
    • Set 268,435,456 (256MB) to min_task_size option.

Last modified: Apr 28 2017 09:59:24 UTC

If this article is incorrect or outdated, or omits critical information, please let us know. For all other issues, please see our support channels.