Skip to main content
In this section, we provide guides and references to use the Hex connector. Configure and schedule Hex metadata and profiler workflows from the Collate UI:

​
Requirements

Hex connector supports both personal and workspace tokens for authentication.

​
Hex Account Setup

Follow the steps below to configure the account setup for Hex connector:

​
Step 1: Generate API Token

To generate an API token in Hex:
  1. Log into your Hex account
  2. Navigate to Account Settings
  3. Go to the API Keys section
  4. Click on “Create API Key”
  5. Choose the token type:
    • Personal Token: Provides access to projects you own or have access to
    • Workspace Token: Provides access to all projects in the workspace (requires admin privileges)
  6. Copy and securely store the generated token
API tokens are shown only once when created. Make sure to copy and store them securely.

​
Step 2: Verify API Access

Ensure that the token has appropriate permissions:
  • For personal tokens: Verify you have access to the projects you want to ingest
  • For workspace tokens: Verify admin privileges are granted

​
Metadata Ingestion

​
Connection Details

1

Connection Details

hostPort: URL to the Hex instance.Provide the URL to your Hex instance. For Hex cloud users, use https://app.hex.tech.For on-premise installations, provide your custom domain URL.tokenType: Type of authentication token.Select the type of token you’re using:
  • personal: Personal access token (default)
  • workspace: Workspace-level token (requires admin privileges)
token: Hex API Token.Provide the API token generated from your Hex account. This token is used for authentication with the Hex APIs.To generate a token:
  • Log into Hex
  • Go to Account Settings > API Keys
  • Create a new API key
  • Copy the generated token
includeTags: Import Hex Categories and Status as Tags.Enable this option to import Hex project categories and status as OpenMetadata tags. This helps in organizing and filtering your Hex projects within OpenMetadata.By default, this is set to true.Dashboard Filter Pattern:Use regex patterns to include or exclude specific dashboards from ingestion. This helps you control which Hex projects are imported into OpenMetadata.Examples:
  • Include only production projects: ^prod-.*
  • Exclude test projects: ^(?!test-).*
2

Test the Connection

Once the credentials have been added, click on Test Connection and Save the changes.Test Connection
3

Configure Metadata Ingestion

After adding and testing the dashboard service, configure the metadata ingestion pipeline. To configure, follow the steps below:
  1. Navigate to Settings > Services > Dashboards.
  2. On the Dashboards services page, click the service you’ve added.
  3. Go to the Agents tab, and then click Add Agent > Add Metadata Agent.
  4. Configure the ingestion details. See Metadata Ingestion Options. Configure Metadata Ingestion

​
Metadata Ingestion Options

  • Name: This field is the name of the ingestion pipeline. Customize it or use the generated name.
  • Dashboard Filter Pattern (Optional): Use it to control whether to include dashboards as part of metadata ingestion.
    • Include: Explicitly include dashboards by adding comma-separated regular expressions to the ‘Include’ field. OpenMetadata will include all dashboards with names matching one or more of the supplied regular expressions. All other dashboards will be excluded.
    • Exclude: Explicitly exclude dashboards by adding comma-separated regular expressions to the ‘Exclude’ field. OpenMetadata will exclude all dashboards with names matching one or more of the supplied regular expressions. All other dashboards will be included.
  • Chart Filter Pattern (Optional): Use it to control whether to include charts as part of metadata ingestion.
    • Include: Explicitly include charts by adding comma-separated regular expressions to the ‘Include’ field. OpenMetadata will include all charts with names matching one or more of the supplied regular expressions. All other charts will be excluded.
    • Exclude: Explicitly exclude charts by adding comma-separated regular expressions to the ‘Exclude’ field. OpenMetadata will exclude all charts with names matching one or more of the supplied regular expressions. All other charts will be included.
  • Data Model Filter Pattern (Optional): Use it to control whether to include data models as part of metadata ingestion.
    • Include: Explicitly include data models by adding comma-separated regular expressions to the ‘Include’ field. OpenMetadata will include all data models with names matching one or more of the supplied regular expressions. All other data models will be excluded.
    • Exclude: Explicitly exclude data models by adding comma-separated regular expressions to the ‘Exclude’ field. OpenMetadata will exclude all data models with names matching one or more of the supplied regular expressions. All other data models will be included.
  • Project Filter Pattern: Filter the dashboards, charts, and data sources by projects. Note that all of them support regex as include or exclude. For example, “My project, My proj.*, .*Project”.
    We filter the projects by concatenating the entire project hierarchy using dot notation (for example, Project1.NestedProjectA.OtherProject). Make sure the regex filter pattern accounts for this fully-qualified format.
  • Enable Debug Log: Enable this toggle to use debug-level logging.
Metadata Ingestion Option 1
  • Lineage Information (Optional): Configure this section to enable lineage between your dashboards and the database tables they are built on. OpenMetadata uses the database service name to match and draw the lineage path from table to dashboard.
    • Db Service Prefixes: Enter one or more service path prefixes to tell OpenMetadata where to look for the source tables used by your dashboards. Supported formats:
      • DBServiceName—matches all tables in the service
      • DBServiceName.DatabaseName—matches tables in a specific database
      • DBServiceName.DatabaseName.SchemaName—matches tables in a specific schema
      • DBServiceName.DatabaseName.SchemaName.TableName—matches a specific table
  • Query Parser Configuration: Controls how OpenMetadata parses SQL queries to extract lineage. Use this to select the SQL parser that best fits your data source’s dialect.
    • Query Parser Type: Choose the SQL parser for lineage extraction:
      • Auto (default): Automatically tries SqlGlot first, falls back to SqlFluff, then SqlParse. Recommended for best results.
      • SqlGlot: High-performance parser with good dialect support. Falls back to SqlParse on failure.
      • SqlFluff: Comprehensive but slower parser with strong dialect support. Falls back to SqlParse on failure.
  • Include Current Owners: Enable this toggle to control whether to include owners for the ingested entity if the owner email matches a user stored in the Collate server as part of metadata ingestion. If the ingested entity already exists and has an owner, the owner will not be overwritten.
  • Mark Deleted Dashboards: Optional configuration to soft delete dashboards in OpenMetadata if the source dashboards are deleted. After deleting, all associated entities like lineage and other related data for that dashboard will be deleted.
  • Mark Deleted Data Models: Optional configuration to soft delete data models in OpenMetadata if the source data models are deleted. After deleting, all associated entities with that data model will be deleted.
  • Mark Deleted Charts: Optional configuration to soft delete charts in OpenMetadata if the source charts are deleted. After deleting, all associated entities with that chart will be deleted.
  • Include Tags: Enable this toggle to control whether to include tags in metadata ingestion.
  • Include Data Models: Enable this toggle to control whether to include data models as part of metadata ingestion.
  • Include Draft Dashboard: Enable this toggle to include draft dashboards. By default, this is enabled.
  • Include Usage: Enable this toggle to control whether to include usage data as part of metadata ingestion.
  • Override Metadata: Enable this toggle to control whether to override the existing metadata in the Collate server with the metadata fetched from the source. If enabled, the metadata fetched from the source will override the existing metadata in OpenMetadata. If disabled, only fields that have no value in OpenMetadata will be updated. This is applicable for fields like description, tags, owner, and displayName.
  • Override Lineage: Enable this toggle to control whether to override the existing lineage in OpenMetadata with the lineage fetched from the source. If enabled, existing lineage will be replaced. If disabled, new lineage edges will be added without removing existing ones.
Metadata Ingestion Option 2
4

Schedule the Ingestion and Deploy

Scheduling can be set up at an hourly, daily, weekly, or manual cadence. The timezone is in UTC. Select a Start Date to schedule for ingestion. It is optional to add an End Date.Review your configuration settings. If they match what you intended, click Deploy to create the service and schedule metadata ingestion.If something doesn’t look right, click the Back button to return to the appropriate step and change the settings as needed.After configuring the workflow, you can click on Deploy to create the pipeline.Schedule the Workflow
5

View the Ingestion Pipeline

Once the workflow has been successfully deployed, you can view the Ingestion Pipeline running from the Service Page.View Ingestion Pipeline
If AutoPilot is enabled, workflows like usage tracking, data lineage, and similar tasks will be handled automatically. Users don’t need to set up or manage them - AutoPilot takes care of everything in the system.
When using a Hybrid Ingestion Runner, any sensitive credential fields—such as passwords, API keys, or private keys—must reference secrets using the following format:
password: secret:/my/database/password
This applies only to fields marked as secrets in the connection form (these typically mask input and show a visibility toggle icon). For a complete guide on managing secrets in hybrid setups, see the Hybrid Ingestion Runner Secret Management Guide.

​
Lineage

Hex connector extracts lineage information by:
  • Warehouse Query History: Directly queries data warehouse logs (Snowflake, BigQuery, Databricks, Redshift, etc.) to find queries executed by Hex
  • Hex Metadata Identification: Identifies Hex-originated queries using metadata comments embedded by Hex containing project IDs
  • SQL Parsing: Analyzes the discovered SQL queries to extract upstream table dependencies
The lineage information helps you understand:
  • Which data warehouse tables are actually queried by each Hex project
  • Real data usage patterns based on executed queries
  • Impact analysis when making changes to underlying data sources
Note: The connector does not use Hex APIs for lineage. Instead, it discovers lineage by analyzing actual queries executed in your data warehouses, providing more accurate data dependency information.

​
Warehouse Query Log Requirements

For lineage extraction to work, query logging must be enabled in your data warehouse:

​
Snowflake

  • Requires access to SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY
  • Query history is automatically maintained by Snowflake

​
BigQuery

  • Requires access to INFORMATION_SCHEMA.JOBS_BY_PROJECT
  • Query history is automatically maintained by BigQuery

​
MySQL

  • Requires general_log to be enabled:
SET GLOBAL general_log = 'ON';
SET GLOBAL log_output = 'TABLE';
  • Grant access to mysql.general_log table

​
Databricks

  • Requires access to system.query.history
  • Query history is automatically maintained

​
Redshift

  • Requires access to stl_query system table
  • Query history is automatically maintained

​
Other Warehouses

  • Athena: Access to query history via CloudTrail or S3 logs
  • ClickHouse: Access to system.query_log table

​
Troubleshooting

Hex Troubleshooting

Learn more about how to troubleshoot common Hex connector issues and resolve configuration or ingestion errors.