> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getcollate.io/llms.txt
> Use this file to discover all available pages before exploring further.

# BYOC Configuration Reference

> Reference for the Collate AI Proxy Helm values that control bot authentication, model tiers, context retrieval, logging, and error reporting.

# Configuration Reference

The [Deployment](/how-to-guides/deployment/byoc/kubernetes/collate-ai-proxy/deployment) page covers the minimum values needed to bring the Collate AI Proxy up against a provider. This page documents the remaining `config` values: bot authentication, model tiers, context retrieval, message limits, logging, and error reporting.

The `config` values in this reference map to environment variables that the chart generates. The settings in [Settings without a chart value](#settings-without-a-chart-value) use `extraEnvs`, so the chart does not add them until you supply them.

<Note>
  These settings require a Collate AI Proxy chart release whose values schema includes the `config` keys on this page, running an AI platform image of **1.13.3 or later**. The published chart repository does not yet contain a compatible release. Until it does, do not add these values to an older chart: Helm rejects unknown `config` keys during schema validation rather than ignoring them.

  Check the chart version you are about to install:

  ```shell theme={null}
  helm repo update
  helm search repo collate-ai-proxy/collate-ai-proxy --versions | head
  ```
</Note>

## Bot authentication and impersonation

The proxy has no user database of its own. When an agent needs metadata, the proxy calls back into the Collate REST API as the `CollateAIApplicationBot` and passes the requesting user's identity, so Collate applies that user's roles and policies to every read. The bot must have impersonation enabled in Collate for this to work.

How the proxy obtains the bot token depends on `config.secretsManager.provider`.

### Static token

With the default `noop` provider, the proxy uses the token you supply directly. Retrieve it from Collate under **Settings** > **Bots** > **CollateAIApplicationBot**.

For production, keep the token in a Kubernetes Secret and reference it. When `botJwtTokenSecretRef.name` and `botJwtTokenSecretRef.key` are both set, the secret reference takes precedence over the inline `botJwtToken`.

```shell theme={null}
kubectl create secret generic caip-bot-token \
  --from-literal=bot-jwt="<COLLATE_AI_APPLICATION_BOT_JWT>" \
  -n <collate-namespace>
```

```yaml theme={null}
config:
  client:
    botJwtTokenSecretRef:
      name: "caip-bot-token"
      key: "bot-jwt"
  secretsManager:
    provider: "noop"
```

### Secrets manager

With a managed provider, the proxy reads the token from your secrets store at runtime and refreshes it on a five minute cache, so rotating the bot token does not require a redeploy. The pod needs credentials for that store, through IRSA, pod identity, workload identity, or an equivalent.

The proxy lowercases the prefix, cluster name, and bot name before it builds the secret path:

```
[prefix] / <clusterName> / bot / <botName> / config / jwttoken
```

```yaml theme={null}
config:
  clusterName: "production"
  secretsManager:
    provider: "managed-aws"
    botName: "CollateAIApplicationBot"
    parameters:
      region: "<AWS_REGION_CODE>"
```

That example resolves to `/production/bot/collateaiapplicationbot/config/jwttoken` in AWS Secrets Manager, and `config.client.botJwtToken` is not needed.

| Provider                                        | Backend                 | Reads the secret itself | Required parameters      |
| ----------------------------------------------- | ----------------------- | ----------------------- | ------------------------ |
| `noop`                                          | None                    | No                      | -                        |
| `managed-aws`                                   | AWS Secrets Manager     | Yes                     | `region`                 |
| `managed-aws-ssm`                               | AWS SSM Parameter Store | Yes                     | `region`                 |
| `managed-azure-kv`                              | Azure Key Vault         | Yes                     | `vaultName`              |
| `gcp`                                           | GCP Secret Manager      | Yes                     | `projectId`              |
| `kubernetes`                                    | Kubernetes Secrets API  | Yes                     | `namespace`, `inCluster` |
| `aws`, `aws-ssm`, `azure-kv`, `db`, `in-memory` | None                    | No                      | -                        |

<Warning>
  `aws` and `managed-aws` are not the same setting. Only the `managed-` providers, along with `gcp` and `kubernetes`, fetch the token from the store. The rest behave like `noop`: they expect the token to already be present in `config.client.botJwtToken` or its secret reference, and the proxy cannot authenticate to Collate if it is empty.
</Warning>

Path separators differ per backend. AWS joins the lowercased segments with `/`; GCP, Azure Key Vault, and Kubernetes join them with `-`. Kubernetes expects the token in a Secret data key named `value`; GCP returns the Secret Manager payload, and Azure Key Vault returns the secret value directly.

| Value                                        | Environment variable             | Default                   | Description                                                                              |
| -------------------------------------------- | -------------------------------- | ------------------------- | ---------------------------------------------------------------------------------------- |
| `config.client.botJwtToken`                  | `BOT_JWT_TOKEN`                  | `""`                      | Bot JWT used to authenticate to Collate. Required unless a managed provider supplies it. |
| `config.client.botJwtTokenSecretRef.name`    | -                                | `""`                      | Kubernetes Secret holding the bot JWT.                                                   |
| `config.client.botJwtTokenSecretRef.key`     | -                                | `""`                      | Key inside that Secret.                                                                  |
| `config.clusterName`                         | `OPENMETADATA_CLUSTER_NAME`      | `openmetadata`            | Cluster segment of the secret path.                                                      |
| `config.secretsManager.provider`             | `SECRET_MANAGER`                 | `noop`                    | Secrets manager backend.                                                                 |
| `config.secretsManager.prefix`               | `SECRETS_MANAGER_PREFIX`         | `""`                      | Optional prefix prepended to the secret path.                                            |
| `config.secretsManager.botName`              | `SECRETS_MANAGER_BOT_NAME`       | `CollateAIApplicationBot` | Bot segment of the secret path.                                                          |
| `config.secretsManager.parameters.region`    | `SECRETS_MANAGER_REGION`         | `""`                      | AWS region for `managed-aws` and `managed-aws-ssm`.                                      |
| `config.secretsManager.parameters.vaultName` | `SECRETS_MANAGER_VAULT_NAME`     | `""`                      | Key Vault name for `managed-azure-kv`.                                                   |
| `config.secretsManager.parameters.projectId` | `SECRETS_MANAGER_PROJECT_ID`     | `""`                      | Project ID for `gcp`.                                                                    |
| `config.secretsManager.parameters.namespace` | `SECRETS_MANAGER_K8S_NAMESPACE`  | `default`                 | Namespace holding the Secret for `kubernetes`.                                           |
| `config.secretsManager.parameters.inCluster` | `SECRETS_MANAGER_K8S_IN_CLUSTER` | `false`                   | Use in-cluster credentials for `kubernetes`.                                             |

## Model tiers

The proxy routes work across three model tiers, each configurable on its own.

* `model` handles general work: planner routing, workers, and chat.
* `modelSmall` handles light tasks such as result titles and entity profiling.
* `modelAdvanced` is used for SQL generation only.

`modelAdvanced` takes a comma-separated list tried in order. If every entry is unavailable, SQL generation falls back to `model`. Setting it to an empty string sends SQL generation to `model` directly, which is the way to roll back a model change without redeploying anything else.

```yaml theme={null}
config:
  llmProvider:
    model: "anthropic.claude-sonnet-4-5-20250929-v1:0"
    modelSmall: "anthropic.claude-haiku-4-5-20251001-v1:0"
    modelAdvanced: "anthropic.claude-opus-4-8,anthropic.claude-opus-4-7"
    maxIteration: 30
    dynamicMaxIterations: 100
```

| Value                                     | Environment variable               | Default                                               | Description                                       |
| ----------------------------------------- | ---------------------------------- | ----------------------------------------------------- | ------------------------------------------------- |
| `config.llmProvider.model`                | `LLM_PROVIDER_MODEL_TYPE`          | `anthropic.claude-sonnet-4-5-20250929-v1:0`           | General purpose model.                            |
| `config.llmProvider.modelSmall`           | `LLM_PROVIDER_MODEL_TYPE_SMALL`    | `anthropic.claude-haiku-4-5-20251001-v1:0`            | Model for light tasks.                            |
| `config.llmProvider.modelAdvanced`        | `LLM_PROVIDER_MODEL_TYPE_ADVANCED` | `anthropic.claude-opus-4-8,anthropic.claude-opus-4-7` | Ordered model list for SQL generation.            |
| `config.llmProvider.maxIteration`         | `LLM_MAX_ITERATIONS`               | `30`                                                  | Reasoning iterations before the agent is stopped. |
| `config.llmProvider.dynamicMaxIterations` | `LLM_DYNAMIC_MAX_ITERATIONS`       | `100`                                                 | Iteration ceiling when dynamic cutoffs apply.     |

The defaults are Bedrock model identifiers. If you run a different provider, set all three tiers to identifiers that provider recognizes, and confirm each one is enabled for your account before rolling out.

## Azure OpenAI authentication mode

Azure OpenAI deployments authenticate with an API key by default. Set `authType` to use the pod's Microsoft Entra workload identity instead, which avoids storing a key in the release.

Before you enable this mode, configure AKS with an OIDC issuer and Microsoft Entra Workload ID, create a federated credential for the service account, and grant the managed identity the **Cognitive Services OpenAI User** role on the Azure OpenAI resource. The workload-identity webhook injects `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, and `AZURE_FEDERATED_TOKEN_FILE` only into pods labeled `azure.workload.identity/use: "true"`.

Use the same service account namespace and name in the federated credential subject, then configure the chart and proxy together:

```yaml theme={null}
serviceAccount:
  create: true
  name: "caip-workload-identity"
  annotations:
    azure.workload.identity/client-id: "<AZURE_CLIENT_ID>"
podLabels:
  azure.workload.identity/use: "true"

config:
  llmProvider:
    type: "openai"
    openAI:
      azureOpenAI:
        enabled: true
        authType: "workload_identity"
        apiVersion: "<AZURE_OPENAI_API_VERSION>"
        deploymentName: "<AZURE_OPENAI_DEPLOYMENT_NAME>"
        resourceName: "<AZURE_OPENAI_RESOURCE_NAME>"
```

| Value                                            | Environment variable         | Default   | Description                           |
| ------------------------------------------------ | ---------------------------- | --------- | ------------------------------------- |
| `config.llmProvider.openAI.azureOpenAI.authType` | `LLM_AZURE_OPENAI_AUTH_TYPE` | `api_key` | Authentication mode for Azure OpenAI. |

## Message and tool result limits

`maxUserMessageChars` rejects user messages above the limit. The truncation limits cap how much of a tool result reaches the model, which is what keeps a large query result from consuming the whole context window. Results the proxy classifies as large payloads get the higher ceiling.

```yaml theme={null}
config:
  validation:
    maxUserMessageChars: 8000
  toolResultTruncation:
    defaultMaxChars: 20000
    largePayloadMaxChars: 200000
```

| Value                                              | Environment variable                  | Default  | Description                                           |
| -------------------------------------------------- | ------------------------------------- | -------- | ----------------------------------------------------- |
| `config.validation.maxUserMessageChars`            | `MAX_USER_MESSAGE_CHARS`              | `8000`   | Maximum characters accepted in a user message.        |
| `config.toolResultTruncation.defaultMaxChars`      | `TOOL_RESULT_DEFAULT_MAX_CHARS`       | `20000`  | Characters retained from a tool result.               |
| `config.toolResultTruncation.largePayloadMaxChars` | `TOOL_RESULT_LARGE_PAYLOAD_MAX_CHARS` | `200000` | Characters retained from a large payload tool result. |

## Context retrieval

`vectorSearchContextThreshold` is the minimum relevance score for a result to enter context, so raising it filters out more results and lowering it widens the set. Raising `vectorSearchContextSize` or `maxContextItems` raises the respective candidate or final-context cap and can increase token usage per request.

```yaml theme={null}
config:
  client:
    vectorSearchContextThreshold: 0.55
    vectorSearchContextSize: 40
    maxContextItems: 25
```

| Value                                        | Environment variable              | Default | Description                                                                                     |
| -------------------------------------------- | --------------------------------- | ------- | ----------------------------------------------------------------------------------------------- |
| `config.client.vectorSearchContextThreshold` | `VECTOR_SEARCH_CONTEXT_THRESHOLD` | `0.55`  | Minimum relevance score for a result to enter context.                                          |
| `config.client.vectorSearchContextSize`      | `VECTOR_SEARCH_CONTEXT_SIZE`      | `40`    | Number of candidates retrieved from vector search before the final context-item cap is applied. |
| `config.client.maxContextItems`              | `MAX_CONTEXT_ITEMS`               | `25`    | Maximum context items kept after deduplication and processing.                                  |

## Logging

`level` sets the root log level. `baseBuilderLogLevel` targets the prompt builder alone, so you can turn up prompt construction detail without raising the volume of every other logger.

```yaml theme={null}
config:
  logging:
    level: "INFO"
    baseBuilderLogLevel: "DEBUG"
```

| Value                                | Environment variable     | Default | Description                              |
| ------------------------------------ | ------------------------ | ------- | ---------------------------------------- |
| `config.logging.level`               | `LOG_LEVEL`              | `INFO`  | Root log level for the proxy.            |
| `config.logging.baseBuilderLogLevel` | `BASE_BUILDER_LOG_LEVEL` | `INFO`  | Log level for prompt builder operations. |

## Error reporting

The proxy can send errors and traces to your own Sentry project. It is disabled by default and stays off unless you set a DSN.

Keep the DSN in a Kubernetes Secret rather than in the values file. When `dsnSecretRef.name` and `dsnSecretRef.key` are both set, the secret reference takes precedence over the inline `dsn`.

```yaml theme={null}
config:
  sentry:
    enabled: true
    dsnSecretRef:
      name: "caip-sentry"
      key: "sentry-dsn"
    environment: "production"
    serverName: "ai-platform"
    tracesSampleRate: 0.5
```

| Value                                 | Environment variable             | Default       | Description                               |
| ------------------------------------- | -------------------------------- | ------------- | ----------------------------------------- |
| `config.sentry.enabled`               | `SENTRY_ENABLED`                 | `false`       | Enables Sentry reporting.                 |
| `config.sentry.dsn`                   | `SENTRY_DSN`                     | `""`          | Sentry DSN.                               |
| `config.sentry.dsnSecretRef.name`     | -                                | `""`          | Kubernetes Secret holding the DSN.        |
| `config.sentry.dsnSecretRef.key`      | -                                | `""`          | Key inside that Secret.                   |
| `config.sentry.environment`           | `SENTRY_ENVIRONMENT`             | `development` | Environment name attached to events.      |
| `config.sentry.serverName`            | `SENTRY_SERVER_NAME`             | `ai-platform` | Server name attached to events.           |
| `config.sentry.tracesSampleRate`      | `SENTRY_TRACES_SAMPLE_RATE`      | `0.5`         | Fraction of traces sent, between 0 and 1. |
| `config.sentry.debug`                 | `SENTRY_DEBUG`                   | `false`       | Verbose Sentry client logging.            |
| `config.sentry.release`               | `SENTRY_RELEASE`                 | `""`          | Release identifier attached to events.    |
| `config.sentry.captureSpanAttributes` | `SENTRY_CAPTURE_SPAN_ATTRIBUTES` | `false`       | Include span attributes in events.        |

## Settings without a chart value

A few proxy settings have no dedicated chart value. Set them through `extraEnvs`, which the chart appends after its generated container environment.

```yaml theme={null}
extraEnvs:
  - name: COLLATE_API_VERSION
    value: "v1"
```

<Warning>
  `VECTOR_SEARCH_K` is bound into the AI platform configuration but the current company-context and hybrid-search paths do not consume it. Setting it does not change the number of candidates retrieved, so do not use it as a retrieval-tuning control.
</Warning>

| Environment variable    | Default | Description                                                                        |
| ----------------------- | ------- | ---------------------------------------------------------------------------------- |
| `COLLATE_API_VERSION`   | `v1`    | Collate REST API version the proxy calls.                                          |
| `VECTOR_SEARCH_K`       | `1000`  | Reserved configuration value. It is not currently consumed by the retrieval paths. |
| `LLM_MAX_OUTPUT_TOKENS` | `32000` | Maximum output tokens per model call. Available in AI platform 1.13.4 and later.   |

## Navigation

<CardGroup cols={2}>
  <Card title="Validation" href="/how-to-guides/deployment/byoc/kubernetes/collate-ai-proxy/validation">
    \< Previous
  </Card>

  <Card title="Additional settings" href="/how-to-guides/deployment/byoc/kubernetes/collate-ai-proxy/additional-settings">
    Next >
  </Card>
</CardGroup>