Hybrid Runner Configuration References

Environment Variables Reference

The Hybrid Runner supports two ways to pass environment variables to ingestion pods.

Setting Environment Variables

Via config.ingestionPods.customConfig.containerParams.env: Use containerParams to set environment variables on ingestion pods.

config:
  ingestionPods:
    customConfig:
      enabled: true
      containerParams:
        env:
          - name: HTTP_PROXY
            value: http://corp-proxy.svc:8080
          - name: HTTPS_PROXY
            value: http://corp-proxy.svc:8080
          - name: NO_PROXY
            value: ".svc, .mycorp.internal"

Runner Environment Variables

DYNAMIC_INGESTION_VERSION_ENABLED: When enabled, the Runner automatically resolves the ingestion pod image tag to match your Collate server version. Only disable this if you mirror Collate images to your own private registry and manage versioning manually:
extraEnvs: - name: DYNAMIC_INGESTION_VERSION_ENABLED value: 'false'

Note: It is recommended to keep DYNAMIC_INGESTION_VERSION_ENABLED value to true. This ensures ingestion fixes and updates are applied automatically without having to manage image tags yourself.

Advanced Configuration

Use the options below to customise workflow behaviour, configure container settings, and run multiple Hybrid Runner instances in the same cluster.

Defining Workflow Fields

Override workflow-level and container-level fields using custom configuration.

Workflow Parameters (workflowParams) Use workflowParams to override workflow-level fields. A typical use case is defining tolerations or pod affinity/anti-affinity:

  config:
    ingestionPods:
      customConfig:
        enabled: true
        workflowParams:
          tolerations:
            - key: team
              effect: NoSchedule
              operator: Equal
              value: data-science

Container Parameters (containerParams) Use containerParams to override container-level fields such as environment variables:

  config:
    ingestionPods:
      customConfig:
        enabled: true
        containerParams:
          env:
            - name: HTTP_PROXY
              value: http://corp-proxy.svc:8080
            - name: HTTPS_PROXY
              value: http://corp-proxy.svc:8080
            - name: NO_PROXY
              value: ".svc, .mycorp.internal"

Note: Custom workflow fields require Hybrid Runner Helm chart version 1.12.5 or later.

Adding Pod Labels

Set pod labels on ingestion pods for pod security policies, cost attribution, or workload identification.

Note: Pod labeling requires Hybrid Runner Helm chart version 1.12.9 or later.

Argo Workflows Executor

Set pod labels via config.ingestionPods.customConfig.workflowParams:

config:
  ingestionPods:
    customConfig:
      enabled: true
      workflowParams:
        podMetadata:
          labels:
            app.kubernetes.io/name: collate-hybrid-ingestion-runner-ingestion
            app.kubernetes.io/part-of: collate-hybrid-ingestion-runner

Simple Kubernetes Executor

Use a podSpecFilePath override file to set labels and annotations on ingestion pod templates. Create a YAML file with a metadata block alongside any other pod spec overrides:

metadata:
  labels:
    cost-allocation/team: data-platform
    my.test/label: verified
  annotations:
    my.test/annotation: "true"
tolerations:
  - key: dedicated
    operator: Equal
    value: ingestion
    effect: NoSchedule

Then reference the file in your Helm values:

config:
  ingestionPods:
    podSpecFilePath: /path/to/pod-spec-overrides.yaml

After you upgrade the runner to 1.12.9, your pod labels will appear on all ingestion pods automatically.

Running Multiple Instances in the Same Cluster

Argo Workflows is a cluster-wide application. When deploying multiple Hybrid Runner instances in a single cluster, only one instance should install Argo Workflows. For each additional instance, set:

installArgoWorkflows: false
config:
  argoWorkflows:
    endpoint: http://argo-workflows-server.argo-workflows.svc:2746

Configuring Node Scheduling for Ingestion Pods

By default, ingestion pods are scheduled on any available node in your cluster. If your cluster uses node taints to isolate workloads, you must configure tolerations and node affinity so ingestion pods can be scheduled on the correct nodes.

If every node in your cluster has a NoSchedule taint and no tolerations are configured, ingestion pods will fail to schedule and remain stuck in Pending state. This is the most common cause of scheduling failures after deploying the Hybrid Runner.

To check the taints on your nodes:

kubectl get nodes -o=custom-columns=NAME:.metadata.name,TAINTS:.spec.taints

Choosing the Right Configuration Key

The configuration key depends on which executor your Hybrid Runner uses:

Executor	Environment variable	When to use
Argo Workflows (default)	`ARGO_PIPELINE_TYPE_CONFIGS`	Hybrid Runner deployed with `installArgoWorkflows: true`
Simple Kubernetes	`SIMPLEK8S_PIPELINE_TYPE_CONFIGS`	Hybrid Runner deployed without Argo Workflows

Do not set both environment variables. Each executor reads from its own configuration key. Setting SIMPLEK8S_PIPELINE_TYPE_CONFIGS when using Argo has no effect, and vice versa.

Argo Workflows Executor

Add the following to your values.yaml. Replace openmetadata-hybrid-runner with the taint value used in your cluster:

extraEnvs:
  - name: ARGO_PIPELINE_TYPE_CONFIGS
    value: >-
      {
        "automation": {
          "toleration": "openmetadata-hybrid-runner",
          "affinity": {
            "nodeAffinity": {
              "requiredDuringSchedulingIgnoredDuringExecution": {
                "nodeSelectorTerms": [
                  {
                    "matchExpressions": [
                      {
                        "key": "nodetype",
                        "operator": "In",
                        "values": ["openmetadata-hybrid-runner"]
                      }
                    ]
                  }
                ]
              }
            }
          }
        },
        "metadata": {
          "toleration": "openmetadata-hybrid-runner"
        },
        "profiler": {
          "toleration": "openmetadata-hybrid-runner"
        },
        "lineage": {
          "toleration": "openmetadata-hybrid-runner"
        }
      }

The toleration field accepts a single string value. The runner creates a fixed dedicated=<value> Kubernetes toleration for all ingestion pods of that type. For full Kubernetes toleration objects (multiple taints, custom operators or effects), use config.ingestionPods.customConfig.workflowParams instead.

Simple Kubernetes Executor

If you’re using the Simple Kubernetes executor without Argo Workflows, use SIMPLEK8S_PIPELINE_TYPE_CONFIGS instead:

extraEnvs:
  - name: SIMPLEK8S_PIPELINE_TYPE_CONFIGS
    value: >-
      {
        "automation": {
          "tolerations": [
            {
              "key": "nodetype",
              "operator": "Equal",
              "value": "openmetadata-hybrid-runner",
              "effect": "NoSchedule"
            }
          ],
          "nodeSelector": {
            "nodetype": "openmetadata-hybrid-runner"
          }
        }
      }

Supported Configuration Fields

Each pipeline type (automation, metadata, profiler, lineage) supports the following fields:

Field	Description
`toleration`	A single taint value string. The runner creates a `dedicated=<value>` Kubernetes toleration for ingestion pods of that type. For full toleration control, use `config.ingestionPods.customConfig.workflowParams`.
`affinity`	Node and pod affinity rules — supports `nodeAffinity`, `podAffinity`, and `podAntiAffinity`
`nodeSelector`	Key-value labels to target specific nodes
`priorityClass`	Kubernetes priority class name for the pod
`resources`	CPU and memory `requests` and `limits`

Configuring Ingestion Pod Memory

Ingestion and automation jobs (including Test Connection) run in pods that carry memory requests and limits. The environment variables that set these values differ by executor, so you must use the names that match the executor your Hybrid Runner runs. Set only the variables that match your executor: on a Simple Kubernetes runner, the ARGO_WORKFLOW_MEMORY_* variables are present in the pod environment but don’t control the submitted Job’s memory, and the reverse is true on an Argo runner. Confirm your executor first (see Choosing the Right Configuration Key).

Memory Variables by Executor

Executor	Memory request	Memory limit	Default request / limit
Argo Workflows (default)	`ARGO_WORKFLOW_MEMORY_REQUEST`	`ARGO_WORKFLOW_MEMORY_LIMIT`	`256Mi` / `4096Mi`
Simple Kubernetes	`INGESTION_REQUEST_MEMORY`	`INGESTION_LIMIT_MEMORY`	Inherited from the runner defaults

Set the variables through extraEnvs. For an Argo Workflows runner:

extraEnvs:
  - name: ARGO_WORKFLOW_MEMORY_REQUEST
    value: '4Gi'
  - name: ARGO_WORKFLOW_MEMORY_LIMIT
    value: '4Gi'

For a Simple Kubernetes runner:

extraEnvs:
  - name: INGESTION_REQUEST_MEMORY
    value: '4Gi'
  - name: INGESTION_LIMIT_MEMORY
    value: '4Gi'

Clusters That Require Equal Requests and Limits

Some clusters enforce an admission policy that requires requests.memory to equal limits.memory, for example a Kyverno memory-requests-equal-limits policy. The default Argo values (256Mi request, 4096Mi limit) are unequal, so on these clusters the admission webhook rejects every ingestion and Test Connection Job before it starts. Set the request and limit to the same value so the Job satisfies the policy:

extraEnvs:
  # Argo Workflows executor
  - name: ARGO_WORKFLOW_MEMORY_REQUEST
    value: '4Gi'
  - name: ARGO_WORKFLOW_MEMORY_LIMIT
    value: '4Gi'

Use INGESTION_REQUEST_MEMORY and INGESTION_LIMIT_MEMORY instead if your runner uses the Simple Kubernetes executor. For how to recognize and diagnose this rejection, see Test Connection or Ingestion Job Rejected in the troubleshooting guide.

Prometheus Metrics

The Hybrid Runner exposes operational metrics in a Prometheus-compatible format via an HTTP endpoint. These metrics provide insight into agent state, activity, and performance.

The available metrics may evolve over time. Inspect the /metrics endpoint directly for the latest set of available metrics.

Configuration

Configure the metrics endpoint in your Helm values:

metricsServerConfiguration:
  port: ${METRICS_SERVER_PORT:-8989}
  path: ${METRICS_SERVER_PATH:-/metrics}

port: Port on which the metrics endpoint is served (default: 8989).
path: HTTP path for accessing metrics (default: /metrics).

Both parameters support environment variable overrides.

Accessing Metrics

Once configured, access metrics at:

http://<agent-host>:<port>/<path>

With default settings:

http://localhost:8989/metrics

Example Metric

# HELP collate_hybrid_agent_connected Is the agent connected to the server?
# (0 = No, 1 = Yes)
# TYPE collate_hybrid_agent_connected gauge
collate_hybrid_agent_connected 1.0

This metric indicates whether the agent is currently connected to the Collate server — 1.0 means connected, 0 means disconnected.

Hybrid Runner Images

This section covers how to host Collate images in your own container registry and how to manage image tags when you prefer to control versioning yourself.

Hosting Your Own Docker Images

To mirror Collate images to your own container registry (for example, Google Artifact Registry), set the following values. Mirror both the Hybrid Runner image and the Ingestion pod image — both are required for the Runner to function correctly.

Hybrid Runner image: Sets the repository, tag, and pull credentials for the main Runner pod:

image:
  repository: my-repo.com/my-image
  tag: my-tag
imagePullSecrets: my-credentials

Ingestion pod image: Sets the repository, tag, and pull credentials for the pods that execute ingestion jobs:

config:
  ingestionPods:
    repository: my-repo.com/my-image
    tag: my-tag
    imagePullSecrets: my-credentials

Managing Your Own Image Tags

By default, the Hybrid Runner automatically resolves image tags to match your Collate server version (for example, om-2.0.5-cl-2.0.5). To manage your own tags, disable automatic resolution using the DYNAMIC_INGESTION_VERSION_ENABLED environment variable. For configuration details, see Runner Environment Variables.

​Environment Variables Reference

​Setting Environment Variables

​Runner Environment Variables

​Advanced Configuration

​Defining Workflow Fields

​Adding Pod Labels

​Argo Workflows Executor

​Simple Kubernetes Executor

​Running Multiple Instances in the Same Cluster

​Configuring Node Scheduling for Ingestion Pods

​Choosing the Right Configuration Key

​Argo Workflows Executor

​Simple Kubernetes Executor

​Supported Configuration Fields

​Configuring Ingestion Pod Memory

​Memory Variables by Executor

​Clusters That Require Equal Requests and Limits

​Prometheus Metrics

​Configuration

​Accessing Metrics

​Example Metric

​Hybrid Runner Images

​Hosting Your Own Docker Images

​Managing Your Own Image Tags

Environment Variables Reference

Setting Environment Variables

Runner Environment Variables

Advanced Configuration

Defining Workflow Fields

Adding Pod Labels

Argo Workflows Executor

Simple Kubernetes Executor

Running Multiple Instances in the Same Cluster

Configuring Node Scheduling for Ingestion Pods

Choosing the Right Configuration Key

Argo Workflows Executor

Simple Kubernetes Executor

Supported Configuration Fields

Configuring Ingestion Pod Memory

Memory Variables by Executor

Clusters That Require Equal Requests and Limits

Prometheus Metrics

Configuration

Accessing Metrics

Example Metric

Hybrid Runner Images

Hosting Your Own Docker Images

Managing Your Own Image Tags