Deployment

This section describes how to deploy the Collate AI Proxy using the official Helm chart. It assumes that the prerequisites have already been completed. You can find all available configuration options in the public values.yaml file. If you need to integrate another provider or use specific models within a supported provider, contact the Collate team for configuration and benchmarking guidance.

Supported providers

Provider	Best for	Primary cloud	Status	Chat models	Embedding models
AWS Bedrock	AWS customers	AWS	Default and recommended	Sonnet 4.5, Haiku 4.5	Amazon Titan
OpenAI	Direct OpenAI access	Any	1.12 release	GPT-4o	`text-embedding-3-small`
Azure OpenAI	Azure customers	Azure	1.12 release	GPT-4o	`text-embedding-3-small`
Google Gemini	GCP customers	GCP	1.12 release	`gemini-2.5-pro`	Configured in the Collate application

Provider configuration

AWS
Azure
GCP

This section defines the settings you need to deploy the Collate AI Proxy using AWS Bedrock as the provider.

Create the Helm values file

The following is the minimum required configuration:

imagePullSecrets:
  - name: omd-registry-credentials

collate:
  hostAndPort: http://openmetadata:8585

config:
  llmProvider:
    bedrock:
      awsRegion: <AWS_REGION_CODE>

serviceAccount:
  create: false
  name: om-role

extraEnvs:
- name: AWS_DEFAULT_REGION
  value: <AWS_REGION_CODE>

If your organization exposes Amazon Bedrock through an API gateway, private endpoint, or another custom URL, set config.llmProvider.bedrock.baseUrl in your Helm values to that endpoint.

config:
  llmProvider:
    bedrock:
      awsRegion: <AWS_REGION_CODE>
      baseUrl: "https://<your-bedrock-endpoint>"

Keep awsRegion set to the Bedrock region even when you override the endpoint with baseUrl.

imagePullSecrets: The secret with the credentials to pull the Docker images.
collate.hostAndPort: The Collate Kubernetes DNS name. Since both components run in the same namespace, it matches the pattern http://<collate_service_name>:<collate_http_service_port>.
config.llmProvider.bedrock.awsRegion: The AWS region where Bedrock is available.
config.llmProvider.bedrock.baseUrl: Optional custom Bedrock endpoint for API gateway, private endpoint, or other managed URL setups.
serviceAccount.create: Set to false to reuse an existing service account.
serviceAccount.name: Set this to the Collate server service account to simplify the post-deployment steps.

Set up IAM role permissions

First, identify the service account used by your Collate server, updating the deployment name and namespace as needed:

kubectl get deployment openmetadata -n namespace -o jsonpath='{ .spec.template.spec.serviceAccount }{"\n"}'

The IAM role assumed by the service account needs access to AWS Bedrock. Add the following statement to its policy:

{
  "Statement": [
    {
      "Action": "bedrock:InvokeModel*",
      "Effect": "Allow",
      "Resource": [
        "arn:aws:bedrock:*::foundation-model/*",
        "arn:aws:bedrock:*:*:inference-profile/*"
      ],
      "Sid": "BedrockInferenceProfileAccess"
    }
  ],
  "Version": "2012-10-17"
}

If you are using IRSA, follow the AWS guide. If you are using pod identity, follow the corresponding AWS guide instead.

Install the Helm chart

Add the Collate AI Proxy Helm chart repository to your local Helm configuration, refresh the chart index, and deploy the chart into the target namespace:

helm repo add collate-ai-proxy https://open-metadata.github.io/collate-ai-proxy-helm-chart
helm repo update
helm install caip collate-ai-proxy/collate-ai-proxy --values values.yaml -n <collate-namespace>

This section defines the settings you need to deploy the Collate AI Proxy using Azure OpenAI as the provider.

Create the Helm values file

The following is the minimum required configuration:

replicaCount: 1

imagePullSecrets:
  - name: omd-registry-credentials

collate:
  hostAndPort: "http://openmetadata:8585"

config:
  llmProvider:
    type: "openai"
    model: "gpt-4o"
    modelSmall: "gpt-4o"
    openAI:
      apiKey: "<AZURE_OPENAI_API_KEY>"
      baseUrl: "<AZURE_OPENAI_BASE_URL>"
      azureOpenAI:
        enabled: true
        apiVersion: "<AZURE_OPENAI_API_VERSION>"
        deploymentName: "<AZURE_OPENAI_DEPLOYMENT_NAME>"
        resourceName: "<AZURE_OPENAI_RESOURCE_NAME>"

If your Azure OpenAI deployment is exposed through Azure API Management or another custom URL, set config.llmProvider.openAI.baseUrl to that endpoint in your Helm values. This lets Collate AI Platform send requests to the managed endpoint instead of the default Azure resource URL.

config:
  llmProvider:
    openAI:
      baseUrl: "https://<your-azure-openai-endpoint>"
      azureOpenAI:
        enabled: true
        apiVersion: "<AZURE_OPENAI_API_VERSION>"
        deploymentName: "<AZURE_OPENAI_DEPLOYMENT_NAME>"
        resourceName: "<AZURE_OPENAI_RESOURCE_NAME>"

When you use a custom Azure endpoint, keep the Azure-specific fields populated so the proxy can route requests to the correct deployment.

Look up the apiVersion in the Azure OpenAI Foundry portal for the selected deployment.

Install the Helm chart

Add the Collate AI Proxy Helm chart repository to your local Helm configuration, refresh the chart index, and deploy the chart into the target namespace:

helm repo add collate-ai-proxy https://open-metadata.github.io/collate-ai-proxy-helm-chart
helm repo update
helm install caip collate-ai-proxy/collate-ai-proxy --values values.yaml -n <collate-namespace>

This section defines the settings you need to deploy the Collate AI Proxy using Google Gemini as the provider.

Create the Helm values file

The following is the minimum required configuration:

replicaCount: 1

imagePullSecrets:
  - name: omd-registry-credentials

collate:
  hostAndPort: "http://openmetadata:8585"

config:
  llmProvider:
    # Type identifier of the configured LLM provider (e.g., bedrock, openai, anthropic, google, ollama).
    type: "google"
    # Provider-specific model identifier.
    model: "gemini-2.5-pro"
    # Provider-specific identifier for the smaller model variant.
    modelSmall: "gemini-2.5-flash"
    google:
      # API key used for Google Generative AI.
      apiKey: ""
      # Secret reference for Google Generative AI API key; takes precedence over apiKey when name and key are both set.
      apiKeySecretRef:
        # Kubernetes Secret resource name.
        name: ""
        # Key inside the Secret that stores the Google API key.
        key: ""
      # Base URL for Google Generative AI API calls.
      baseUrl: "https://generativelanguage.googleapis.com"

Use either config.llmProvider.google.apiKey or config.llmProvider.google.apiKeySecretRef.For production deployments, Collate recommends using config.llmProvider.google.apiKeySecretRef so the API key stays in a Kubernetes Secret managed by your cluster secret workflow. When both apiKeySecretRef.name and apiKeySecretRef.key are set, the secret reference takes precedence over the inline apiKey.

Configure the Gemini API key secret

If you manage Kubernetes secrets through an external secrets provider, create or sync a secret in the Collate namespace, then reference that secret in your Helm values:

config:
  llmProvider:
    google:
      apiKeySecretRef:
        name: "gemini-api-key"
        key: "api-key"

If you manage the secret directly with kubectl, create a generic Kubernetes Secret:

kubectl create secret generic gemini-api-key \
  --from-literal=api-key="<GOOGLE_GENERATIVE_AI_API_KEY>" \
  -n <collate-namespace>

Then reference the same secret in your Helm values:

config:
  llmProvider:
    google:
      apiKeySecretRef:
        name: "gemini-api-key"
        key: "api-key"

Install the Helm chart

Add the Collate AI Proxy Helm chart repository to your local Helm configuration, refresh the chart index, and deploy the chart into the target namespace:

helm repo add collate-ai-proxy https://open-metadata.github.io/collate-ai-proxy-helm-chart
helm repo update
helm install caip collate-ai-proxy/collate-ai-proxy --values values.yaml -n <collate-namespace>

Prerequisites

< Previous

Configure Collate

Next >

​Deployment

​Supported providers

​Provider configuration

​Create the Helm values file

​Set up IAM role permissions

​Install the Helm chart

​Create the Helm values file

​Install the Helm chart

​Create the Helm values file

​Configure the Gemini API key secret

​Install the Helm chart

Prerequisites

Configure Collate

Deployment

Supported providers

Provider configuration

Create the Helm values file

Set up IAM role permissions

Install the Helm chart

Create the Helm values file

Install the Helm chart

Create the Helm values file

Configure the Gemini API key secret

Install the Helm chart