> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getcollate.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Hybrid Runner Deployment on Azure (AKS)

> Step-by-step guide to deploying the Collate Hybrid Ingestion Runner on Azure Kubernetes Service using Helm.

## Azure (AKS) Architecture Diagram

<img
  src="https://mintcdn.com/collatedocs/DnroXjZcyOK_PA9U/public/images/deployment/kubernetes/azure-hybrid-runner-architecture.svg?fit=max&auto=format&n=DnroXjZcyOK_PA9U&q=85&s=6c6b9d1f96cc76ba894ee890eb5f753e"
  alt="Azure AKS Hybrid Runner architecture diagram showing the WebSocket connection
between Collate SaaS and the Hybrid Runner, and the flow from Argo Workflows
through Ingestion Pods to Azure Key Vault"
  width="760"
  height="320"
  data-path="public/images/deployment/kubernetes/azure-hybrid-runner-architecture.svg"
/>

## Prerequisites

Before you deploy on Azure (AKS), confirm the following:

* **ECR credentials**: You've received AWS ECR (Elastic Container Registry)
  credentials (accessKeyId and secretAccessKey) from Collate to pull the
  required Docker images.

* **Kubernetes cluster**: You have an AKS (Azure Kubernetes Service) cluster
  running Kubernetes 1.28+.

* **Tooling**: `helm` and `kubectl` are installed and configured to target
  your cluster.

* **Secrets store**: A secrets store is configured for your cluster. Store
  your connector credentials in the secrets store — they can't be entered as
  plain text in the Collate UI.

  By default, Kubernetes Secrets are used and the Helm chart installs all
  required RBAC (Role-Based Access Control) automatically. To use Azure Key
  Vault, see [Secrets Management](#secrets-management).

* **Authentication token**: You have a JWT (JSON Web Token) from the
  IngestionBot.

  For more information about how to get your Collate JWT, see
  [Obtain Collate JSON Web Token (JWT)](/how-to-guides/deployment/hybrid-runner/requirements#obtain-collate-json-web-token-jwt).

  <Note>
    **Note**: This token doesn't expire. Store it securely and treat it as a
    secret.
  </Note>

## Installation Procedure

The Helm chart bundles all required Kubernetes resources:

* The Hybrid Runner Server pod
* Argo Workflows (installed as a dependency)
* A cron job to periodically renew credentials from Collate's ECR registry

Helm chart repository:
[hybrid-ingestion-runner-helm-chart](https://github.com/open-metadata/hybrid-ingestion-runner-helm-chart)

### Step 1: Add the Helm Repository

Add the Collate Helm repository and fetch the latest chart index:

```bash theme={null}
helm repo add collate-hybrid https://open-metadata.github.io/hybrid-ingestion-runner-helm-chart
helm repo update
```

### Step 2: Create the Helm Values File

Create a `values.yaml` file with the minimum required configuration. Replace
the placeholder values with the credentials provided by Collate and your
instance details:

```yaml theme={null}
config:
  agentId: "azure-prod"         # Descriptive name shown in the Collate UI
  authToken: <Collate IngestionBot JWT token>
  serverHost: <your-instance>.getcollate.io

ecrRegistryHelper:
  collateCredentials:
    values:
      accessKeyId: <Provided by Collate>
      secretAccessKey: <Provided by Collate>

installArgoWorkflows: true
```

### Step 3: Create the Argo Workflows Namespace

Create a dedicated namespace for Argo Workflows before running the Helm
install:

```bash theme={null}
kubectl create namespace argo-workflows
```

### Step 4: Deploy the Helm Chart

Install the Hybrid Runner chart using the values file you created in Step 2:

```bash theme={null}
helm upgrade --install collate-prod collate-hybrid/hybrid-ingestion-runner --namespace argo-workflows --values values.yaml
```

<Note>
  **Note**: If you run the above command more than once, Helm hooks may not
  re-execute and the ECR registry helper may fail to generate credentials. If
  the Runner pod shows `ImagePullBackOff`, trigger the credentials helper
  manually:

  ```bash theme={null}
  kubectl create job --from=cronjob/ecr-registry-helper manual
  ```
</Note>

## Verify the Deployment

Once the Helm chart is deployed, run the following checks to confirm everything
is working correctly.

* **Check Pod Running Status**:

  Confirm all pods are running before proceeding:

  ```bash theme={null}
  kubectl get pods
  ```

* **Confirm Runner Shows Active in the Collate UI**:

  1. Navigate to **Settings** > **Preferences** > **Ingestion Runners**.
  2. Find your runner by its agentId (for example, `azure-prod`).
  3. Confirm the status shows **Active**.

* **Verify Argo Workflows Pods Are Running**:

  Confirm the Argo Workflows controller and server pods are healthy:

  ```bash theme={null}
  kubectl get pods -n argo-workflows -l app.kubernetes.io/part-of=argo-workflows
  ```

* **Confirm ECR Cron Job Health**:

  Confirm the ECR credentials cron job is present and scheduled:

  ```bash theme={null}
  kubectl get cronjob ecr-registry-helper
  ```

## Creating a Service With the Hybrid Runner

Use this procedure to add a new service and connect it to your Hybrid Runner.

1. Navigate to **Settings** > **Services** and select the service type you
   want to add.
2. Select the service connector and select **Next**.
3. Enter the **Service Name** and a **Description** (optional), then click
   **Next**.
4. In the **Ingestion Runner** drop-down, select your Hybrid Runner.
5. Fill in the connection details. For any credential field that shows a
   hide/show toggle, enter the secret reference path from your secrets store
   instead of the actual value:

   ```
     secret:<secret-name>
   ```

   **Important:** Your credentials must be stored in your secrets store before
   referencing them here. See [Secrets Management](#secrets-management).

## Secrets Management

The Hybrid Runner supports two approaches for managing connector credentials.
Kubernetes Secrets is the default and requires no additional configuration.

### Option 1: Kubernetes Secrets (Default)

The Helm chart uses native Kubernetes Secrets and automatically installs the
required RBAC (Role-Based Access Control) permissions on your cluster. This
works for most customers without any additional setup.

**Storing a Secret**:

Create a Kubernetes Secret in the same namespace as your Helm release (defaults to `default`):

```bash theme={null}
kubectl create secret generic my-db-password \
  --from-literal=value='your-secret-value'
```

If you deployed the Helm chart to a custom namespace, add `-n <your-namespace>` to the command.

### Option 2: Azure Key Vault (Optional)

Configure **Workload Identity** with a User Assigned Managed Identity to grant
the ingestion pods access to Azure Key Vault. The service account name is
`ingestion` by default.

**Required role:** `Key Vault Secrets Officer`

Update your Helm values to configure the secrets manager, annotate the pod
service account, and label the Argo Workflows pods for Workload Identity:

```yaml theme={null}
config:
  secretsManager: "managed-azure-kv"
  ingestionPods:
    serviceAccount:
      annotations:
        "azure.workload.identity/client-id": <user_assigned_managed_identity_client_id>
    extraEnvs:
      - "AZURE_KEY_VAULT_NAME:<azure_key_vault_name>"

argoWorkflows:
  controller:
    workflowDefaults:
      spec:
        podMetadata:
          labels:
            azure.workload.identity/use: "true"
```

Replace `<user_assigned_managed_identity_client_id>` with your managed
identity's client ID and `<azure_key_vault_name>` with the name of your
Key Vault.

**Storing a Secret**:

In the Azure Portal, navigate to your **Key Vault** > **Secrets** >
**Generate/Import**. Set the secret name and paste the value as-is.

### Referencing a Secret in the Collate UI

For both options, enter the secret reference path in any masked credential
field (fields with a hide/show toggle) in the connection form:

```
secret:<secret-name>
```

For example:

```
secret:my-db-password
```

<Note>
  Secret references only work for masked fields—those with a hide/show toggle
  icon.
</Note>