> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getcollate.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Databricks Troubleshooting

> Troubleshoot Databricks connector issues including schema extraction errors or driver failures.

## Troubleshooting

### Workflow Deployment Error

If there were any errors during the workflow deployment process, the
Ingestion Pipeline Entity will still be created, but no workflow will be
present in the Ingestion container.

* You can then Edit the Ingestion Pipeline and **Deploy** it again.
* From the Connection tab, you can also Edit the Service if needed.

## Connector Debug Troubleshooting

This section provides instructions to help resolve common issues encountered during connector setup and metadata ingestion in Collate. Below are some of the most frequently observed troubleshooting scenarios.

## How to Enable Debug Logging for Any Ingestion

To enable debug logging for any ingestion workflow in Collate:

1. **Navigate to Connections**
   Go to **Connections** > **Service Type** (e.g., Database) in the Collate UI.

2. **Select a Service**
   Choose the specific service for which you want to enable debug logging.

3. **Access Agents Tab**
   Go to the **Agents tab** and click the three-dot menu on the right-hand side of the ingestion type, and select Edit.

4. **Enable Debug Logging**
   In the configuration dialog, enable the **Debug Log** option and click **Next**.

5. **Schedule and Submit**
   Configure the schedule if needed and click **Submit** to apply the changes.

## Permission Issues

If you encounter permission-related errors during connector setup or metadata ingestion, ensure that all the prerequisites and access configurations specified for each connector are properly implemented. Refer to the connector-specific documentation to verify the required permissions.

## Databricks Connection Details

```
source:
  type: databricks
  serviceName: local_databricks
  serviceConnection:
    config:
      catalog: hive_metastore
      databaseSchema: default
      token: <databricks token>
      hostPort: localhost:443
      connectionArguments:
        http_path: <http path of databricks cluster>
  sourceConfig:
    config:
      type: DatabaseMetadata
sink:
  type: metadata-rest
  config: {}
workflowConfig:
  openMetadataServerConfig:
    hostPort: http://localhost:8585/api
    authProvider: no-auth
```

Here are the steps to get `hostPort`, `token`, and `http_path`.

First, log in to Azure Databricks and, from the sidebar, select SQL Warehouse (in the SQL section).

<img src="https://mintcdn.com/collatedocs/12mtkhFtvTP7FUxZ/public/images/connectors/databricks/select-sql-warehouse.png?fit=max&auto=format&n=12mtkhFtvTP7FUxZ&q=85&s=a6ccc6fecaf022d45aa2e7e93786a49a" alt="Select Sql Warehouse" width="1848" height="1358" data-path="public/images/connectors/databricks/select-sql-warehouse.png" />

Now, click on SQL Warehouse from the SQL Warehouses list.

<img src="https://mintcdn.com/collatedocs/12mtkhFtvTP7FUxZ/public/images/connectors/databricks/Open-sql-warehouse.png?fit=max&auto=format&n=12mtkhFtvTP7FUxZ&q=85&s=5d6e2843334b4577ee5823062b0200c5" alt="Open Sql Warehouse" width="1420" height="1402" data-path="public/images/connectors/databricks/Open-sql-warehouse.png" />

On that page, go to the Connection Details section.
Server hostname and Port together form your `hostPort` value, and HTTP path is your `http_path` value.

<img src="https://mintcdn.com/collatedocs/12mtkhFtvTP7FUxZ/public/images/connectors/databricks/Connection-details.png?fit=max&auto=format&n=12mtkhFtvTP7FUxZ&q=85&s=b88e4860e18859b972730990dd569aa6" alt="Connection details" width="2860" height="1338" data-path="public/images/connectors/databricks/Connection-details.png" />

In the Connection Details section, click on Create a personal access token.

<img src="https://mintcdn.com/collatedocs/12mtkhFtvTP7FUxZ/public/images/connectors/databricks/Open-create-tocken-page.png?fit=max&auto=format&n=12mtkhFtvTP7FUxZ&q=85&s=a4146387838f6299efde74d9e45aed4d" alt="Open create token" width="2860" height="1338" data-path="public/images/connectors/databricks/Open-create-tocken-page.png" />

On this page, create a new `token`.

<img src="https://mintcdn.com/collatedocs/12mtkhFtvTP7FUxZ/public/images/connectors/databricks/Generate-token.png?fit=max&auto=format&n=12mtkhFtvTP7FUxZ&q=85&s=e2e1ae65869fa2bd98ad67cb6432ead2" alt="Generate token" width="2748" height="1068" data-path="public/images/connectors/databricks/Generate-token.png" />