Configuration Reference

The Deployment page covers the minimum values needed to bring the Collate AI Proxy up against a provider. This page documents the remaining config values: bot authentication, model tiers, context retrieval, message limits, logging, and error reporting. The config values in this reference map to environment variables that the chart generates. The settings in Settings without a chart value use extraEnvs, so the chart does not add them until you supply them.

These settings require a Collate AI Proxy chart release whose values schema includes the config keys on this page, running an AI platform image of 1.13.3 or later. The published chart repository does not yet contain a compatible release. Until it does, do not add these values to an older chart: Helm rejects unknown config keys during schema validation rather than ignoring them.Check the chart version you are about to install:

helm repo update
helm search repo collate-ai-proxy/collate-ai-proxy --versions | head

Bot authentication and impersonation

The proxy has no user database of its own. When an agent needs metadata, the proxy calls back into the Collate REST API as the CollateAIApplicationBot and passes the requesting user’s identity, so Collate applies that user’s roles and policies to every read. The bot must have impersonation enabled in Collate for this to work. How the proxy obtains the bot token depends on config.secretsManager.provider.

Static token

With the default noop provider, the proxy uses the token you supply directly. Retrieve it from Collate under Settings > Bots > CollateAIApplicationBot. For production, keep the token in a Kubernetes Secret and reference it. When botJwtTokenSecretRef.name and botJwtTokenSecretRef.key are both set, the secret reference takes precedence over the inline botJwtToken.

kubectl create secret generic caip-bot-token \
  --from-literal=bot-jwt="<COLLATE_AI_APPLICATION_BOT_JWT>" \
  -n <collate-namespace>

config:
  client:
    botJwtTokenSecretRef:
      name: "caip-bot-token"
      key: "bot-jwt"
  secretsManager:
    provider: "noop"

Secrets manager

With a managed provider, the proxy reads the token from your secrets store at runtime and refreshes it on a five minute cache, so rotating the bot token does not require a redeploy. The pod needs credentials for that store, through IRSA, pod identity, workload identity, or an equivalent. The proxy lowercases the prefix, cluster name, and bot name before it builds the secret path:

[prefix] / <clusterName> / bot / <botName> / config / jwttoken

config:
  clusterName: "production"
  secretsManager:
    provider: "managed-aws"
    botName: "CollateAIApplicationBot"
    parameters:
      region: "<AWS_REGION_CODE>"

That example resolves to /production/bot/collateaiapplicationbot/config/jwttoken in AWS Secrets Manager, and config.client.botJwtToken is not needed.

Provider	Backend	Reads the secret itself	Required parameters
`noop`	None	No	-
`managed-aws`	AWS Secrets Manager	Yes	`region`
`managed-aws-ssm`	AWS SSM Parameter Store	Yes	`region`
`managed-azure-kv`	Azure Key Vault	Yes	`vaultName`
`gcp`	GCP Secret Manager	Yes	`projectId`
`kubernetes`	Kubernetes Secrets API	Yes	`namespace`, `inCluster`
`aws`, `aws-ssm`, `azure-kv`, `db`, `in-memory`	None	No	-

aws and managed-aws are not the same setting. Only the managed- providers, along with gcp and kubernetes, fetch the token from the store. The rest behave like noop: they expect the token to already be present in config.client.botJwtToken or its secret reference, and the proxy cannot authenticate to Collate if it is empty.

Path separators differ per backend. AWS joins the lowercased segments with /; GCP, Azure Key Vault, and Kubernetes join them with -. Kubernetes expects the token in a Secret data key named value; GCP returns the Secret Manager payload, and Azure Key Vault returns the secret value directly.

Value	Environment variable	Default	Description
`config.client.botJwtToken`	`BOT_JWT_TOKEN`	`""`	Bot JWT used to authenticate to Collate. Required unless a managed provider supplies it.
`config.client.botJwtTokenSecretRef.name`	-	`""`	Kubernetes Secret holding the bot JWT.
`config.client.botJwtTokenSecretRef.key`	-	`""`	Key inside that Secret.
`config.clusterName`	`OPENMETADATA_CLUSTER_NAME`	`openmetadata`	Cluster segment of the secret path.
`config.secretsManager.provider`	`SECRET_MANAGER`	`noop`	Secrets manager backend.
`config.secretsManager.prefix`	`SECRETS_MANAGER_PREFIX`	`""`	Optional prefix prepended to the secret path.
`config.secretsManager.botName`	`SECRETS_MANAGER_BOT_NAME`	`CollateAIApplicationBot`	Bot segment of the secret path.
`config.secretsManager.parameters.region`	`SECRETS_MANAGER_REGION`	`""`	AWS region for `managed-aws` and `managed-aws-ssm`.
`config.secretsManager.parameters.vaultName`	`SECRETS_MANAGER_VAULT_NAME`	`""`	Key Vault name for `managed-azure-kv`.
`config.secretsManager.parameters.projectId`	`SECRETS_MANAGER_PROJECT_ID`	`""`	Project ID for `gcp`.
`config.secretsManager.parameters.namespace`	`SECRETS_MANAGER_K8S_NAMESPACE`	`default`	Namespace holding the Secret for `kubernetes`.
`config.secretsManager.parameters.inCluster`	`SECRETS_MANAGER_K8S_IN_CLUSTER`	`false`	Use in-cluster credentials for `kubernetes`.

Model tiers

The proxy routes work across three model tiers, each configurable on its own.

model handles general work: planner routing, workers, and chat.
modelSmall handles light tasks such as result titles and entity profiling.
modelAdvanced is used for SQL generation only.

modelAdvanced takes a comma-separated list tried in order. If every entry is unavailable, SQL generation falls back to model. Setting it to an empty string sends SQL generation to model directly, which is the way to roll back a model change without redeploying anything else.

config:
  llmProvider:
    model: "anthropic.claude-sonnet-4-5-20250929-v1:0"
    modelSmall: "anthropic.claude-haiku-4-5-20251001-v1:0"
    modelAdvanced: "anthropic.claude-opus-4-8,anthropic.claude-opus-4-7"
    maxIteration: 30
    dynamicMaxIterations: 100

Value	Environment variable	Default	Description
`config.llmProvider.model`	`LLM_PROVIDER_MODEL_TYPE`	`anthropic.claude-sonnet-4-5-20250929-v1:0`	General purpose model.
`config.llmProvider.modelSmall`	`LLM_PROVIDER_MODEL_TYPE_SMALL`	`anthropic.claude-haiku-4-5-20251001-v1:0`	Model for light tasks.
`config.llmProvider.modelAdvanced`	`LLM_PROVIDER_MODEL_TYPE_ADVANCED`	`anthropic.claude-opus-4-8,anthropic.claude-opus-4-7`	Ordered model list for SQL generation.
`config.llmProvider.maxIteration`	`LLM_MAX_ITERATIONS`	`30`	Reasoning iterations before the agent is stopped.
`config.llmProvider.dynamicMaxIterations`	`LLM_DYNAMIC_MAX_ITERATIONS`	`100`	Iteration ceiling when dynamic cutoffs apply.

The defaults are Bedrock model identifiers. If you run a different provider, set all three tiers to identifiers that provider recognizes, and confirm each one is enabled for your account before rolling out.

Azure OpenAI authentication mode

Azure OpenAI deployments authenticate with an API key by default. Set authType to use the pod’s Microsoft Entra workload identity instead, which avoids storing a key in the release. Before you enable this mode, configure AKS with an OIDC issuer and Microsoft Entra Workload ID, create a federated credential for the service account, and grant the managed identity the Cognitive Services OpenAI User role on the Azure OpenAI resource. The workload-identity webhook injects AZURE_CLIENT_ID, AZURE_TENANT_ID, and AZURE_FEDERATED_TOKEN_FILE only into pods labeled azure.workload.identity/use: "true". Use the same service account namespace and name in the federated credential subject, then configure the chart and proxy together:

serviceAccount:
  create: true
  name: "caip-workload-identity"
  annotations:
    azure.workload.identity/client-id: "<AZURE_CLIENT_ID>"
podLabels:
  azure.workload.identity/use: "true"

config:
  llmProvider:
    type: "openai"
    openAI:
      azureOpenAI:
        enabled: true
        authType: "workload_identity"
        apiVersion: "<AZURE_OPENAI_API_VERSION>"
        deploymentName: "<AZURE_OPENAI_DEPLOYMENT_NAME>"
        resourceName: "<AZURE_OPENAI_RESOURCE_NAME>"

Value	Environment variable	Default	Description
`config.llmProvider.openAI.azureOpenAI.authType`	`LLM_AZURE_OPENAI_AUTH_TYPE`	`api_key`	Authentication mode for Azure OpenAI.

Message and tool result limits

maxUserMessageChars rejects user messages above the limit. The truncation limits cap how much of a tool result reaches the model, which is what keeps a large query result from consuming the whole context window. Results the proxy classifies as large payloads get the higher ceiling.

config:
  validation:
    maxUserMessageChars: 8000
  toolResultTruncation:
    defaultMaxChars: 20000
    largePayloadMaxChars: 200000

Value	Environment variable	Default	Description
`config.validation.maxUserMessageChars`	`MAX_USER_MESSAGE_CHARS`	`8000`	Maximum characters accepted in a user message.
`config.toolResultTruncation.defaultMaxChars`	`TOOL_RESULT_DEFAULT_MAX_CHARS`	`20000`	Characters retained from a tool result.
`config.toolResultTruncation.largePayloadMaxChars`	`TOOL_RESULT_LARGE_PAYLOAD_MAX_CHARS`	`200000`	Characters retained from a large payload tool result.

Context retrieval

vectorSearchContextThreshold is the minimum relevance score for a result to enter context, so raising it filters out more results and lowering it widens the set. Raising vectorSearchContextSize or maxContextItems raises the respective candidate or final-context cap and can increase token usage per request.

config:
  client:
    vectorSearchContextThreshold: 0.55
    vectorSearchContextSize: 40
    maxContextItems: 25

Value	Environment variable	Default	Description
`config.client.vectorSearchContextThreshold`	`VECTOR_SEARCH_CONTEXT_THRESHOLD`	`0.55`	Minimum relevance score for a result to enter context.
`config.client.vectorSearchContextSize`	`VECTOR_SEARCH_CONTEXT_SIZE`	`40`	Number of candidates retrieved from vector search before the final context-item cap is applied.
`config.client.maxContextItems`	`MAX_CONTEXT_ITEMS`	`25`	Maximum context items kept after deduplication and processing.

Logging

level sets the root log level. baseBuilderLogLevel targets the prompt builder alone, so you can turn up prompt construction detail without raising the volume of every other logger.

config:
  logging:
    level: "INFO"
    baseBuilderLogLevel: "DEBUG"

Value	Environment variable	Default	Description
`config.logging.level`	`LOG_LEVEL`	`INFO`	Root log level for the proxy.
`config.logging.baseBuilderLogLevel`	`BASE_BUILDER_LOG_LEVEL`	`INFO`	Log level for prompt builder operations.

Error reporting

The proxy can send errors and traces to your own Sentry project. It is disabled by default and stays off unless you set a DSN. Keep the DSN in a Kubernetes Secret rather than in the values file. When dsnSecretRef.name and dsnSecretRef.key are both set, the secret reference takes precedence over the inline dsn.

config:
  sentry:
    enabled: true
    dsnSecretRef:
      name: "caip-sentry"
      key: "sentry-dsn"
    environment: "production"
    serverName: "ai-platform"
    tracesSampleRate: 0.5

Value	Environment variable	Default	Description
`config.sentry.enabled`	`SENTRY_ENABLED`	`false`	Enables Sentry reporting.
`config.sentry.dsn`	`SENTRY_DSN`	`""`	Sentry DSN.
`config.sentry.dsnSecretRef.name`	-	`""`	Kubernetes Secret holding the DSN.
`config.sentry.dsnSecretRef.key`	-	`""`	Key inside that Secret.
`config.sentry.environment`	`SENTRY_ENVIRONMENT`	`development`	Environment name attached to events.
`config.sentry.serverName`	`SENTRY_SERVER_NAME`	`ai-platform`	Server name attached to events.
`config.sentry.tracesSampleRate`	`SENTRY_TRACES_SAMPLE_RATE`	`0.5`	Fraction of traces sent, between 0 and 1.
`config.sentry.debug`	`SENTRY_DEBUG`	`false`	Verbose Sentry client logging.
`config.sentry.release`	`SENTRY_RELEASE`	`""`	Release identifier attached to events.
`config.sentry.captureSpanAttributes`	`SENTRY_CAPTURE_SPAN_ATTRIBUTES`	`false`	Include span attributes in events.

Settings without a chart value

A few proxy settings have no dedicated chart value. Set them through extraEnvs, which the chart appends after its generated container environment.

extraEnvs:
  - name: COLLATE_API_VERSION
    value: "v1"

VECTOR_SEARCH_K is bound into the AI platform configuration but the current company-context and hybrid-search paths do not consume it. Setting it does not change the number of candidates retrieved, so do not use it as a retrieval-tuning control.

Environment variable	Default	Description
`COLLATE_API_VERSION`	`v1`	Collate REST API version the proxy calls.
`VECTOR_SEARCH_K`	`1000`	Reserved configuration value. It is not currently consumed by the retrieval paths.
`LLM_MAX_OUTPUT_TOKENS`	`32000`	Maximum output tokens per model call. Available in AI platform 1.13.4 and later.

Validation

< Previous

Additional settings

Next >

Deployment

BYOC Configuration Reference

Configuration Reference

Bot authentication and impersonation

Static token

Secrets manager

Model tiers

Azure OpenAI authentication mode

Message and tool result limits

Context retrieval

Logging

Error reporting

Settings without a chart value

Navigation

Validation

Additional settings

​Configuration Reference

​Bot authentication and impersonation

​Static token

​Secrets manager

​Model tiers

​Azure OpenAI authentication mode

​Message and tool result limits

​Context retrieval

​Logging

​Error reporting

​Settings without a chart value

​Navigation

Validation

Additional settings

Configuration Reference

Bot authentication and impersonation

Static token

Secrets manager

Model tiers

Azure OpenAI authentication mode

Message and tool result limits

Context retrieval

Logging

Error reporting

Settings without a chart value

Navigation