Skip to main content

​
Troubleshooting

​
Workflow Deployment Error

If there were any errors during the workflow deployment process, the Ingestion Pipeline Entity will still be created, but no workflow will be present in the Ingestion container.
  • You can then Edit the Ingestion Pipeline and Deploy it again.
  • From the Connection tab, you can also Edit the Service if needed.

​
Connector Debug Troubleshooting

This section provides instructions to help resolve common issues encountered during connector setup and metadata ingestion in Collate. Below are some of the most frequently observed troubleshooting scenarios.

​
How to Enable Debug Logging for Any Ingestion

To enable debug logging for any ingestion workflow in Collate:
  1. Navigate to Connections Go to Connections > Service Type (e.g., Database) in the Collate UI.
  2. Select a Service Choose the specific service for which you want to enable debug logging.
  3. Access Agents Tab Go to the Agents tab and click the three-dot menu on the right-hand side of the ingestion type, and select Edit.
  4. Enable Debug Logging In the configuration dialog, enable the Debug Log option and click Next.
  5. Schedule and Submit Configure the schedule if needed and click Submit to apply the changes.

​
Permission Issues

If you encounter permission-related errors during connector setup or metadata ingestion, ensure that all the prerequisites and access configurations specified for each connector are properly implemented. Refer to the connector-specific documentation to verify the required permissions.

​
Test Connection Passes but No Tables Are Ingested

If ingestion completes but imports 0 tables, check the GetTables step on Test Connection. It still shows as passed, but can carry one of two distinct caveats, each with its own cause and remedy.

​
Caveat: No Schemas Visible

This means no databases could be listed in the target catalog at all, before the connector ever got to checking individual tables. Two distinct causes:
  • The configured schemaFilterPattern (or the catalog itself) excludes every database that exists.
  • The IAM principal used by the connector is missing the Glue/Athena permissions needed to list databases (for example glue:GetDatabases, athena:ListDatabases), or the Lake Formation Describe grant at the database level.

​
Resolution

  1. Review the configured schemaFilterPattern and confirm it isn’t excluding every database in the catalog.
  2. In IAM, confirm the principal used by the connector (a role, or an IAM user if the connector authenticates with an AWS access key) has the Glue/Athena list permissions from the connector’s required policy.
  3. In the AWS Lake Formation console, grant that principal Describe at the database level for the databases it should list.
  4. Re-run Test Connection and check the GetTables caveat again.

​
Caveat: No Readable Tables

This means databases were listed successfully, but the IAM principal used by the connector is missing AWS Lake Formation Describe and Select grants on the target tables. AWS Lake Formation silently filters results it can’t authorize instead of raising an error, so a catalog with missing table-level grants looks identical to a genuinely empty one. Test Connection can’t tell the two apart, so it passes with this caveat instead of failing outright, which would incorrectly reject a legitimately empty or view-only catalog.

​
Resolution

  1. In the AWS Lake Formation console, open Data permissions.
  2. Grant the IAM principal used by the Athena connector (a role, or an IAM user if the connector authenticates with an AWS access key) Describe and Select permissions on each database and table it needs to ingest.
  3. Re-run Test Connection and check the GetTables caveat again. If the catalog genuinely has no tables yet, this caveat is expected and can be ignored.