> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getcollate.io/llms.txt
> Use this file to discover all available pages before exploring further.

# BYOC Deployment

> Deploy the Collate AI Proxy with Helm using AWS Bedrock, Azure OpenAI, or GCP Gemini configuration values.

# Deployment

This section describes how to deploy the Collate AI Proxy using the official Helm chart. It assumes that the prerequisites have already been completed.

You can find all available configuration options in the public [`values.yaml`](https://github.com/open-metadata/collate-ai-proxy-helm-chart/blob/main/charts/collate-ai-proxy/values.yaml) file.

If you need to integrate another provider or use specific models within a supported provider, contact the Collate team for configuration and benchmarking guidance.

## Supported providers

| Provider      | Best for             | Primary cloud | Status                  | Chat models           | Embedding models                      |
| ------------- | -------------------- | ------------- | ----------------------- | --------------------- | ------------------------------------- |
| AWS Bedrock   | AWS customers        | AWS           | Default and recommended | Sonnet 4.5, Haiku 4.5 | Amazon Titan                          |
| OpenAI        | Direct OpenAI access | Any           | 1.12 release            | GPT-4o                | `text-embedding-3-small`              |
| Azure OpenAI  | Azure customers      | Azure         | 1.12 release            | GPT-4o                | `text-embedding-3-small`              |
| Google Gemini | GCP customers        | GCP           | 1.12 release            | `gemini-2.5-pro`      | Configured in the Collate application |

## Provider configuration

<Tabs>
  <Tab title="AWS">
    This section defines the settings you need to deploy the Collate AI Proxy using AWS Bedrock as the provider.

    ### Create the Helm values file

    The following is the minimum required configuration:

    ```yaml theme={null}
    imagePullSecrets:
      - name: omd-registry-credentials

    collate:
      hostAndPort: http://openmetadata:8585

    config:
      llmProvider:
        bedrock:
          awsRegion: <AWS_REGION_CODE>

    serviceAccount:
      create: false
      name: om-role

    extraEnvs:
    - name: AWS_DEFAULT_REGION
      value: <AWS_REGION_CODE>
    ```

    <Note>
      If your organization exposes Amazon Bedrock through an API gateway, private endpoint, or another custom URL, set `config.llmProvider.bedrock.baseUrl` in your Helm values to that endpoint.

      ```yaml theme={null}
      config:
        llmProvider:
          bedrock:
            awsRegion: <AWS_REGION_CODE>
            baseUrl: "https://<your-bedrock-endpoint>"
      ```

      Keep `awsRegion` set to the Bedrock region even when you override the endpoint with `baseUrl`.
    </Note>

    * `imagePullSecrets`: The secret with the credentials to pull the Docker images.
    * `collate.hostAndPort`: The Collate Kubernetes DNS name. Since both components run in the same namespace, it matches the pattern `http://<collate_service_name>:<collate_http_service_port>`.
    * `config.llmProvider.bedrock.awsRegion`: The AWS region where Bedrock is available.
    * `config.llmProvider.bedrock.baseUrl`: Optional custom Bedrock endpoint for API gateway, private endpoint, or other managed URL setups.
    * `serviceAccount.create`: Set to `false` to reuse an existing service account.
    * `serviceAccount.name`: Set this to the Collate server service account to simplify the post-deployment steps.

    ### Set up IAM role permissions

    First, identify the service account used by your Collate server, updating the deployment name and namespace as needed:

    ```shell theme={null}
    kubectl get deployment openmetadata -n namespace -o jsonpath='{ .spec.template.spec.serviceAccount }{"\n"}'
    ```

    The IAM role assumed by the service account needs access to AWS Bedrock. Add the following statement to its policy:

    ```json theme={null}
    {
      "Statement": [
        {
          "Action": "bedrock:InvokeModel*",
          "Effect": "Allow",
          "Resource": [
            "arn:aws:bedrock:*::foundation-model/*",
            "arn:aws:bedrock:*:*:inference-profile/*"
          ],
          "Sid": "BedrockInferenceProfileAccess"
        }
      ],
      "Version": "2012-10-17"
    }
    ```

    If you are using [IRSA](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html), follow the AWS guide. If you are using [pod identity](https://docs.aws.amazon.com/eks/latest/userguide/pod-identities.html), follow the corresponding AWS guide instead.

    ### Install the Helm chart

    Add the Collate AI Proxy Helm chart repository to your local Helm configuration, refresh the chart index, and deploy the chart into the target namespace:

    ```shell theme={null}
    helm repo add collate-ai-proxy https://open-metadata.github.io/collate-ai-proxy-helm-chart
    helm repo update
    helm install caip collate-ai-proxy/collate-ai-proxy --values values.yaml -n <collate-namespace>
    ```
  </Tab>

  <Tab title="Azure">
    This section defines the settings you need to deploy the Collate AI Proxy using Azure OpenAI as the provider.

    ### Create the Helm values file

    The following is the minimum required configuration:

    ```yaml theme={null}
    replicaCount: 1

    imagePullSecrets:
      - name: omd-registry-credentials

    collate:
      hostAndPort: "http://openmetadata:8585"

    config:
      llmProvider:
        type: "openai"
        model: "gpt-4o"
        modelSmall: "gpt-4o"
        openAI:
          apiKey: "<AZURE_OPENAI_API_KEY>"
          baseUrl: "<AZURE_OPENAI_BASE_URL>"
          azureOpenAI:
            enabled: true
            apiVersion: "<AZURE_OPENAI_API_VERSION>"
            deploymentName: "<AZURE_OPENAI_DEPLOYMENT_NAME>"
            resourceName: "<AZURE_OPENAI_RESOURCE_NAME>"
    ```

    <Note>
      If your Azure OpenAI deployment is exposed through Azure API Management or another custom URL, set `config.llmProvider.openAI.baseUrl` to that endpoint in your Helm values. This lets Collate AI Platform send requests to the managed endpoint instead of the default Azure resource URL.

      ```yaml theme={null}
      config:
        llmProvider:
          openAI:
            baseUrl: "https://<your-azure-openai-endpoint>"
            azureOpenAI:
              enabled: true
              apiVersion: "<AZURE_OPENAI_API_VERSION>"
              deploymentName: "<AZURE_OPENAI_DEPLOYMENT_NAME>"
              resourceName: "<AZURE_OPENAI_RESOURCE_NAME>"
      ```

      When you use a custom Azure endpoint, keep the Azure-specific fields populated so the proxy can route requests to the correct deployment.
    </Note>

    Look up the `apiVersion` in the Azure OpenAI Foundry portal for the selected deployment.

    ### Install the Helm chart

    Add the Collate AI Proxy Helm chart repository to your local Helm configuration, refresh the chart index, and deploy the chart into the target namespace:

    ```shell theme={null}
    helm repo add collate-ai-proxy https://open-metadata.github.io/collate-ai-proxy-helm-chart
    helm repo update
    helm install caip collate-ai-proxy/collate-ai-proxy --values values.yaml -n <collate-namespace>
    ```
  </Tab>

  <Tab title="GCP">
    This section defines the settings you need to deploy the Collate AI Proxy using Google Gemini as the provider.

    ### Create the Helm values file

    The following is the minimum required configuration:

    ```yaml theme={null}
    replicaCount: 1

    imagePullSecrets:
      - name: omd-registry-credentials

    collate:
      hostAndPort: "http://openmetadata:8585"

    config:
      llmProvider:
        # Type identifier of the configured LLM provider (e.g., bedrock, openai, anthropic, google, ollama).
        type: "google"
        # Provider-specific model identifier.
        model: "gemini-2.5-pro"
        # Provider-specific identifier for the smaller model variant.
        modelSmall: "gemini-2.5-flash"
        google:
          # API key used for Google Generative AI.
          apiKey: ""
          # Secret reference for Google Generative AI API key; takes precedence over apiKey when name and key are both set.
          apiKeySecretRef:
            # Kubernetes Secret resource name.
            name: ""
            # Key inside the Secret that stores the Google API key.
            key: ""
          # Base URL for Google Generative AI API calls.
          baseUrl: "https://generativelanguage.googleapis.com"
    ```

    <Note>
      Use either `config.llmProvider.google.apiKey` or `config.llmProvider.google.apiKeySecretRef`.

      For production deployments, Collate recommends using `config.llmProvider.google.apiKeySecretRef` so the API key stays in a Kubernetes Secret managed by your cluster secret workflow. When both `apiKeySecretRef.name` and `apiKeySecretRef.key` are set, the secret reference takes precedence over the inline `apiKey`.
    </Note>

    ### Configure the Gemini API key secret

    If you manage Kubernetes secrets through an external secrets provider, create or sync a secret in the Collate namespace, then reference that secret in your Helm values:

    ```yaml theme={null}
    config:
      llmProvider:
        google:
          apiKeySecretRef:
            name: "gemini-api-key"
            key: "api-key"
    ```

    If you manage the secret directly with `kubectl`, create a generic Kubernetes Secret:

    ```shell theme={null}
    kubectl create secret generic gemini-api-key \
      --from-literal=api-key="<GOOGLE_GENERATIVE_AI_API_KEY>" \
      -n <collate-namespace>
    ```

    Then reference the same secret in your Helm values:

    ```yaml theme={null}
    config:
      llmProvider:
        google:
          apiKeySecretRef:
            name: "gemini-api-key"
            key: "api-key"
    ```

    ### Install the Helm chart

    Add the Collate AI Proxy Helm chart repository to your local Helm configuration, refresh the chart index, and deploy the chart into the target namespace:

    ```shell theme={null}
    helm repo add collate-ai-proxy https://open-metadata.github.io/collate-ai-proxy-helm-chart
    helm repo update
    helm install caip collate-ai-proxy/collate-ai-proxy --values values.yaml -n <collate-namespace>
    ```
  </Tab>
</Tabs>

<CardGroup cols={2}>
  <Card title="Prerequisites" href="/how-to-guides/deployment/byoc/kubernetes/collate-ai-proxy/prerequisites">
    \< Previous
  </Card>

  <Card title="Configure Collate" href="/how-to-guides/deployment/byoc/kubernetes/collate-ai-proxy/configure-collate">
    Next >
  </Card>
</CardGroup>