Skip to main content

​
Guide to Deploy Collate Binaries On-Premises

This guide will help you start using Collate Docker Images to run the OpenMetadata Application in an on-premises Kubernetes cluster, connecting with Argo Workflows for running ingestion from the OpenMetadata Application itself.
This guide presumes you have an on-premises Kubernetes cluster already set up. All code snippets assume the collate namespace unless otherwise noted.

​
Architecture

Collate OpenMetadata requires 4 components:
  1. Collate Server
  2. Database — Collate Server stores the metadata in a relational database. We support MySQL or Postgres.
    • MySQL version 8.0.0 or greater
    • Postgres version 12.0 or greater
  3. Search Engine — We support:
    • ElasticSearch 9.3.0
    • OpenSearch 3.4
  4. Workflow Orchestration — We use Argo Workflows as the orchestrator for ingestion pipelines.

​
Sizing Requirements

​
Hardware Requirements

A Kubernetes Cluster with at least 1 Master Node and 3 Worker Nodes is the required configuration. Each Worker Node should have at least:
  • 4 vCPUs
  • 16 GiB Memory
  • 128 GiB Storage capacity
If you want Collate workloads scheduled on dedicated nodes, use Kubernetes taints and tolerations. Collate OpenMetadata supports tolerations via custom Helm values.

​
Software Requirements

  • Collate OpenMetadata supports Kubernetes Cluster version 1.24 or greater.
  • Collate Docker Images are available via private AWS Elastic Container Registry (ECR). The Collate Team will share credentials and steps to configure Kubernetes to pull Docker Images from AWS ECR.
  • For Argo Workflows, Collate OpenMetadata is currently compatible with application version 3.4+.

​
Database Sizing and Capacity

Our recommendation is to configure PostgreSQL. For 100,000 Data Assets and 1,000 Users:
  • 8 vCPUs
  • 64 GiB Memory
  • 256 GiB Storage Capacity
  • 3,500 IOPS storage

​
Search Client Sizing and Capacity

For 100,000 Data Assets and 1,000 Users:
  • 8 vCPUs
  • 64 GiB Memory
  • 256 GiB Storage Capacity

​
Argo Workflows Ingestion Runners

The recommended resources are 4 vCPUs and 16 GiB of Memory.

​
On-Premises Prerequisites

​
Object Storage for Argo Workflows Artifacts

Argo Workflows requires object storage to archive ingestion logs. On-premises deployments can use MinIO as an S3-compatible object store.

​
Deploy MinIO (if you don’t have an existing object store)

helm repo add minio https://charts.min.io/
helm repo update

helm upgrade --install minio minio/minio \
  --namespace minio \
  --create-namespace \
  --set rootUser=admin \
  --set rootPassword=<MINIO_ROOT_PASSWORD> \
  --set persistence.size=50Gi \
  --set service.type=ClusterIP
Create the Argo Workflows artifacts bucket: 
kubectl run minio-client --image=minio/mc --rm -it --restart=Never -- \
  /bin/sh -c "mc alias set local http://minio.minio:9000 admin <MINIO_ROOT_PASSWORD> && mc mb local/argo-artifacts"

​
Create Kubernetes Secret for MinIO Credentials

kubectl create secret generic argo-artifact-credentials \
  --from-literal=accessKey=<MINIO_ACCESS_KEY> \
  --from-literal=secretKey=<MINIO_SECRET_KEY> \
  --namespace argo-workflows

​
Setup AWS ECR

Collate will provide the credentials to pull Docker Images from a private registry located in AWS ECR.

​
Install AWS CLI

Follow the AWS CLI installation guide to install AWS CLI on your machine.

​
Configure AWS Credentials

aws configure --profile ecr-collate
The command will prompt for credentials. The Collate team will securely share these via a 1Password link. Confirm the credentials are correctly set: 
aws configure list --profile ecr-collate

​
Kubernetes Docker Registry Secrets for AWS ECR

kubectl create secret docker-registry ecr-registry-creds \
  --docker-server=118146679784.dkr.ecr.eu-west-1.amazonaws.com \
  --docker-username=AWS \
  --docker-password=$(aws ecr get-login-password --profile ecr-collate) \
  --namespace <<NAMESPACE_NAME>>
Replace <<NAMESPACE_NAME>> with the namespace where you want to deploy Collate OpenMetadata Server. If the namespace does not exist yet, create it with kubectl create namespace <<NAMESPACE_NAME>>.
AWS ECR Token RefreshECR tokens expire after 12 hours. If a pod is rescheduled after 12 hours, you will get an ImagePullBackOff error. Delete the secret and recreate it using the command above.

​
Install Argo Workflows

​
Add Helm Repository

helm repo add argo https://argoproj.github.io/argo-helm
helm repo update

​
Create the Argo Namespace

kubectl create namespace argo-workflows

​
Kubernetes Secret for Argo Workflows DB Credentials

kubectl create secret generic argo-db-credentials \
  --from-literal=username=<DB_USERNAME> \
  --from-literal=password=<DB_PASSWORD> \
  --namespace argo-workflows

​
Create Custom Helm Values for Argo Workflows

Create a file named argo-workflows.values.yml: 
# argo-workflows.values.yml
controller:
  serviceAccount:
    create: true
    name: argo-workflows-controller-sa
  name: workflow-controller
  workflowDefaults:
    spec:
      serviceAccountName: om-role

server:
  serviceAccount:
    create: true
    name: argo-workflows-server-sa
  extraArgs:
    - "--auth-mode=server"
    - "--request-timeout=5m"

persistence:
  archive: true
  postgresql:
    host: <DATABASE_INSTANCE_ENDPOINT>
    database: <DATABASE_NAME>
    tableName: argo_workflows
    userNameSecret:
      name: argo-db-credentials
      key: username
    passwordSecret:
      name: argo-db-credentials
      key: password
    ssl: true
    sslMode: require

useDefaultArtifactRepo: true
useStaticCredentials: true
artifactRepository:
  archiveLogs: true
  s3:
    # MinIO endpoint — replace with your object store endpoint if different
    endpoint: minio.minio:9000
    insecure: true
    bucket: argo-artifacts
    keyFormat: 'workflows/{{workflow.namespace}}/{{workflow.name}}/{{pod.name}}'
    accessKeySecret:
      name: argo-artifact-credentials
      key: accessKey
    secretKeySecret:
      name: argo-artifact-credentials
      key: secretKey
If you are using an existing S3-compatible store (e.g., Ceph, NetApp StorageGRID) instead of MinIO, update endpoint, bucket, and the secret reference to match your environment. Set insecure: false and configure TLS if your store uses HTTPS.
For further customisation, refer to the community Helm chart values.

​
Deploy Argo Workflows

We target application version 3.7.1 using Helm chart version 0.45.23 (Artifact Hub): 
helm upgrade --install argo-workflows argo/argo-workflows \
  --version 0.45.23 \
  --namespace argo-workflows \
  --values argo-workflows.values.yml

​
[Optional] Enable Prometheus Metrics

If you have a Prometheus Application running on your cluster, enable metrics using: 
controller:
  serviceMonitor:
    enabled: true
server:
  serviceMonitor:
    enabled: true
Refer to the official Argo Workflows documentation for further configuration.

​
Install OpenMetadata/Collate

​
Create the Collate Namespace

kubectl create namespace collate

​
Kubernetes Service Account for Ingestion

The OpenMetadata Application communicates with Argo Workflows to dynamically trigger ephemeral pods that run ingestion workloads. Create a dedicated Kubernetes Service Account: 
kubectl create serviceaccount om-role -n collate

​
Create Long-Lived API Token for the ServiceAccount

kubectl apply -n collate -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: om-role.service-account-token
  annotations:
    kubernetes.io/service-account.name: om-role
type: kubernetes.io/service-account-token
EOF

​
Configure Kubernetes Roles for the Service Account

Create a file om-argo-role.yml: 
# om-argo-role.yml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: om-argo-role
  namespace: collate
rules:
  - verbs: [list, watch, create, update, patch, get, delete]
    apiGroups:
      - argoproj.io
    resources:
      - workflows
  - verbs: [list, watch, patch, get]
    apiGroups:
      - ''
    resources:
      - pods/log
      - pods
  - verbs: [list, watch, create, update, patch, get, delete]
    apiGroups:
      - argoproj.io
    resources:
      - cronworkflows
  - verbs: [create, patch]
    apiGroups:
      - argoproj.io
    resources:
      - workflowtaskresults
Apply the role and create the role binding: 
kubectl apply -f om-argo-role.yml

kubectl create rolebinding om-argo-role-binding \
  --role=om-argo-role \
  --serviceaccount=collate:om-role \
  --namespace collate

​
Install OpenMetadata Helm Chart

Create Kubernetes Secrets for the database connection: 
kubectl create secret generic db-credentials \
  --from-literal=password=<<DATABASE_PASSWORD>> \
  --namespace collate
Add the Helm chart repository: 
helm repo add open-metadata https://helm.open-metadata.org/
helm repo update
If you plan to use the DeltaLake connector, the ARGO_INGESTION_IMAGE value should be: 118146679784.dkr.ecr.eu-west-1.amazonaws.com/collate-customers-ingestion-eu-west-1:om-1.12.3-cl-1.12.3
Create a file openmetadata.values.yml: 
# openmetadata.values.yml
replicaCount: 1
openmetadata:
  config:
    elasticsearch:
      host: ${es_host}
      port: ${es_port}
      scheme: ${es_scheme}
      searchType: elasticsearch
      auth:
        enabled: true
        username: ${es_username}
        password:
          secretRef: es-credentials
          secretKey: password
    database:
      host: ${db_host}
      port: ${db_port}
      driverClass: org.postgresql.Driver
      dbScheme: postgresql
      auth:
        username: ${db_user}
        password:
          secretRef: db-credentials
          secretKey: password
      dbParams: "allowPublicKeyRetrieval=true&useSSL=true&serverTimezone=UTC"
    pipelineServiceClientConfig:
      className: "io.collate.pipeline.argo.ArgoServiceClient"
      apiEndpoint: "http://argo-workflows-server.argo-workflows:2746"
      metadataApiEndpoint: "http://openmetadata:8585/api"
      auth:
        enabled: false
image:
  repository: 118146679784.dkr.ecr.eu-west-1.amazonaws.com/collate-customers-eu-west-1
  tag: om-1.12.3-cl-1.12.3
  imagePullPolicy: IfNotPresent
imagePullSecrets:
  - name: ecr-registry-creds
extraEnvs:
  - name: ARGO_NAMESPACE
    value: collate
  - name: ARGO_TOKEN
    valueFrom:
      secretKeyRef:
        name: "om-role.service-account-token"
        key: "token"
  - name: ARGO_INGESTION_IMAGE
    value: "118146679784.dkr.ecr.eu-west-1.amazonaws.com/collate-customers-ingestion-slim-eu-west-1:om-1.12.3-cl-1.12.3"
  - name: ARGO_WORKFLOW_EXECUTOR_SERVICE_ACCOUNT_NAME
    value: om-role
  - name: ARGO_IMAGE_PULL_SECRETS
    value: ecr-registry-creds
  - name: ASSET_UPLOADER_PROVIDER
    value: "s3"
  - name: ASSET_UPLOADER_MAX_FILE_SIZE
    value: "10485760"
  - name: ASSET_UPLOADER_S3_ENDPOINT
    value: "http://minio.minio:9000"
  - name: ASSET_UPLOADER_S3_BUCKET_NAME
    value: "argo-artifacts"
  - name: ASSET_UPLOADER_S3_PREFIX_PATH
    value: "assets/collate"
serviceAccount:
  name: "openmetadata"
Install the Collate OpenMetadata Application: 
helm upgrade --install openmetadata open-metadata/openmetadata \
  --values openmetadata.values.yml \
  --namespace collate

​
[Optional] Enable Prometheus Metrics

Collate Application exposes Prometheus metrics on port 8586. Enable the integration using: 
serviceMonitor:
  enabled: true

​
Post Installation/Upgrade Steps

​
Configure ReIndexing

After installation or upgrade, configure ReIndexing from the OpenMetadata UI. For detailed steps, refer to the OpenMetadata upgrade documentation.

​
Troubleshooting

​
Pods Stuck in Pending State

Check for resource constraints or missing secrets: 
kubectl describe pod -n collate -l app.kubernetes.io/name=openmetadata
SymptomCauseFix
ImagePullBackOffECR secret missing or expiredRecreate ecr-registry-creds with a fresh ECR token
Insufficient cpu / memoryCluster at capacityReduce resources.requests in openmetadata.values.yml or add nodes
Pending on PVCNo default StorageClassSet a default StorageClass or pass explicit storageClass in values

​
Argo Workflows Cannot Connect to Object Storage

Verify the MinIO service is reachable from the argo-workflows namespace: 
kubectl run test --image=curlimages/curl --rm -it --restart=Never -n argo-workflows -- \
  curl -s http://minio.minio:9000/minio/health/live
Expected response: 200 OK. If it fails, check that MinIO is running and the endpoint in argo-workflows.values.yml is correct.

​
Environment Variables for Collate OpenMetadata Argo

Environment NameDescriptionDefault ValueRequired
ARGO_IMAGE_PULL_SECRETSImage Pull Secret Name to pull Docker Images for Ingestion from a Private Registry. Multiple secrets can be supplied comma-separated.Empty StringFalse
ARGO_INGESTION_IMAGEDocker Image and Tag for Ingestion Imagesopenmetadata/ingestion-base:1.4.3True
ARGO_NAMESPACENamespace in which Argo Workflows will be executed. Must match the namespace where OpenMetadata is deployed.argoTrue
ARGO_SERVER_CERTIFICATE_PATHSSL Certificate Path to connect to Argo ServerEmpty StringFalse
ARGO_TEST_CONNECTION_BACKOFF_TIMEBackoff retry time in seconds to test the connection5False
ARGO_TOKENJWT Token to authenticate with Argo Workflow APIEmpty StringTrue
ARGO_WORKFLOW_CPU_LIMITKubernetes CPU Limits for Argo Workflows created with Ingestion1000mFalse
ARGO_WORKFLOW_CPU_REQUESTKubernetes CPU Requests for Argo Workflows created with Ingestion200mFalse
ARGO_WORKFLOW_CUSTOMER_TOLERATIONKubernetes Node Toleration to schedule Ingestion Workflow Pods to specific NodesargoFalse
ARGO_WORKFLOW_EXECUTOR_SERVICE_ACCOUNT_NAMEService Account Name to be used for Argo Workflows for Ingestionom-roleTrue
ARGO_WORKFLOW_MEMORY_LIMITKubernetes Memory Limits for Argo Workflows created with Ingestion4096MiFalse
ARGO_WORKFLOW_MEMORY_REQUESTKubernetes Memory Requests for Argo Workflows created with Ingestion256MiFalse
ASSET_UPLOADER_ENABLEEnable Asset Upload FeatureTrueFalse
ASSET_UPLOADER_PROVIDERAsset Upload Provider Name. Can be s3 or azure.s3False
ASSET_UPLOADER_MAX_FILE_SIZEMax File Size to support for Asset Upload (in bytes)5242880False
ASSET_UPLOADER_S3_ENDPOINTCustom S3-compatible endpoint (e.g. MinIO)Empty StringFalse
ASSET_UPLOADER_S3_BUCKET_NAMEAsset Upload S3/MinIO Bucket NameEmpty StringFalse
ASSET_UPLOADER_S3_PREFIX_PATHAsset Upload S3/MinIO Prefix Pathassets/defaultFalse