Skip to main content

Troubleshooting

Workflow Deployment Error

If there were any errors during the workflow deployment process, the Ingestion Pipeline Entity will still be created, but no workflow will be present in the Ingestion container.
  • You can then Edit the Ingestion Pipeline and Deploy it again.
  • From the Connection tab, you can also Edit the Service if needed.

Connector Debug Troubleshooting

This section provides instructions to help resolve common issues encountered during connector setup and metadata ingestion in Collate. Below are some of the most frequently observed troubleshooting scenarios.

How to Enable Debug Logging for Any Ingestion

To enable debug logging for any ingestion workflow in Collate:
  1. Navigate to Services Go to Settings > Services > Service Type (e.g., Database) in the Collate UI.
  2. Select a Service Choose the specific service for which you want to enable debug logging.
  3. Access Ingestion Tab Go to the Ingestion tab and click the three-dot menu on the right-hand side of the ingestion type, and select Edit.
  4. Enable Debug Logging In the configuration dialog, enable the Debug Log option and click Next.
  5. Schedule and Submit Configure the schedule if needed and click Submit to apply the changes.

Permission Issues

If you encounter permission-related errors during connector setup or metadata ingestion, ensure that all the prerequisites and access configurations specified for each connector are properly implemented. Refer to the connector-specific documentation to verify the required permissions.

Connection Failed

If you encounter connection errors when connecting to Informix:
Error: [Informix][Informix ODBC Driver]Connection could not be established
This indicates that Collate cannot reach the Informix server or the connection parameters are incorrect.

Resolution

  1. Verify Host and Port: Ensure the hostname and port are correct (e.g., localhost:9088).
  2. Check Server Name: Verify that the Server Name matches the value in your sqlhosts file or INFORMIXSERVER environment variable.
  3. Network Access: Ensure the system running the ingestion can reach the Informix server on the specified port.
  4. Firewall Rules: Check that the Informix port (typically 9088) is not blocked by firewalls.
  5. Server Status: Verify that the Informix server is running using onstat - command.

Authentication Failed

If you encounter authentication errors:
Error: [Informix][Informix ODBC Driver]Incorrect password or user <username> is not known on the database server
This indicates the username or password is incorrect, or the user doesn’t exist.

Resolution

  1. Verify Credentials: Double-check that the username and password are correct.
  2. User Exists: Confirm the user exists in the Informix database:
    SELECT username FROM sysusers WHERE username = '<username>';
    
  3. Password Reset: If needed, reset the password using:
    ALTER USER '<username>' WITH PASSWORD '<new_password>';
    

Permission Denied on System Catalogs

If you encounter permission errors when extracting metadata:
Error: [Informix][Informix ODBC Driver]No SELECT permission on systables
This indicates the user doesn’t have the required permissions on system catalog tables.

Resolution

Grant the necessary permissions on system catalogs:
-- Grant SELECT on system catalog tables
GRANT SELECT ON systables TO '<username>';
GRANT SELECT ON syscolumns TO '<username>';
GRANT SELECT ON sysviews TO '<username>';
GRANT SELECT ON sysprocedures TO '<username>';

Database Does Not Exist

If the specified database cannot be found:
Error: [Informix][Informix ODBC Driver]Database '<database_name>' not found
This indicates the database name is incorrect or the database doesn’t exist.

Resolution

  1. Verify Database Name: Check that the database name is spelled correctly and exists:
    SELECT name FROM sysdatabases;
    
  2. Database Access: Ensure the user has access to the database:
    GRANT CONNECT TO '<username>' ON '<database_name>';
    

SSL Connection Failed

If you encounter SSL connection errors:
Error: [Informix][Informix ODBC Driver]SSL connection failed
This indicates SSL is enabled but the configuration is incorrect.

Resolution

  1. Verify SSL Mode: Ensure the SSL Mode is set correctly (require, verify-ca, or verify-full).
  2. Check CA Certificate: For verify-ca or verify-full modes, ensure the CA certificate is correctly provided.
  3. Server SSL Configuration: Verify that the Informix server is configured for SSL connections.
  4. Certificate Format: Ensure certificates are in PEM format.

SQLTRACE Access Denied

If you encounter errors when extracting lineage or usage:
Error: [Informix][Informix ODBC Driver]No SELECT permission on sysmaster:syssqltrace
This indicates the user doesn’t have permission to access the SQL trace table.

Resolution

  1. Grant Permission: Grant SELECT on the SQL trace table:
    GRANT SELECT ON sysmaster:syssqltrace TO '<username>';
    
  2. Enable SQL Tracing: Ensure SQL tracing is enabled (run as DBA):
    ONMODE -Y 2
    
  3. Verify Trace Status: Check that SQL tracing is active:
    SELECT * FROM sysmaster:syssqltrace LIMIT 1;
    

SQLTRACE Table Too Large

If the syssqltrace table grows too large and impacts performance:
Warning: syssqltrace table has exceeded recommended size

Resolution

  1. Flush Old Entries: Periodically truncate the SQL trace table after successful ingestion:
    -- Disable SQL tracing
    ONMODE -Y 0
    -- Truncate will happen automatically when tracing is re-enabled
    -- Re-enable SQL tracing
    ONMODE -Y 2
    
  2. Lower Trace Level: Consider using SQLTRACE level 1 instead of 2 if full coverage isn’t required.
  3. Schedule Regular Cleanup: Set up a scheduled task to flush the trace table periodically.

Connection Timeout

If you encounter timeout errors:
Error: [Informix][Informix ODBC Driver]Connection timeout
This indicates the connection to Informix took too long to establish.

Resolution

  1. Check Network Latency: Verify network connectivity between the ingestion system and Informix server.
  2. Server Load: Check if the Informix server is under heavy load using onstat -g act.
  3. Increase Timeout: If using a slow network, consider increasing connection timeout settings.
  4. Server Resources: Ensure the Informix server has adequate resources (CPU, memory, disk I/O).

No Tables Found

If the ingestion completes but no tables are found:
Warning: No tables found in database
This may occur if the database is empty or all tables are filtered out.

Resolution

  1. Verify Database Contents: Confirm the database contains tables:
    SELECT tabname FROM systables WHERE tabtype = 'T';
    
  2. Check Table Filters: Review the Table Filter Pattern to ensure it’s not excluding all tables.
  3. User Permissions: Verify the user has SELECT access to the tables you want to ingest.