> ## Documentation Index > Fetch the complete documentation index at: https://docs.getcollate.io/llms.txt > Use this file to discover all available pages before exploring further. # BYOC Prerequisites > Review the software, access, hardware, AWS Bedrock, Azure OpenAI, and GCP Gemini requirements before deploying the Collate AI Proxy with Helm. # Prerequisites Your team must include individuals with strong expertise in Kubernetes and your chosen cloud provider. Before deploying the Collate AI Proxy, ensure you have the following requirements in place. ## Required Software ### Kubernetes Cluster A running Kubernetes cluster, version `1.19` or later. Verify your cluster version with: ```shell theme={null} kubectl version --client ``` ### Helm 3 The Helm package manager for Kubernetes. Download from the [Helm installation guide](https://helm.sh/docs/intro/install/). Verify the installation: ```shell theme={null} helm version ``` ### kubectl The Kubernetes command-line tool. Download from the [kubectl installation guide](https://kubernetes.io/docs/tasks/tools/). Verify the installation: ```shell theme={null} kubectl cluster-info ``` ### Existing Collate Installation An existing Collate deployment must already be running in your cluster. * The Collate AI Proxy (CAIP) communicates with the Collate server over gRPC. * Any communication from CAIP to Collate uses the user's Personal Access Token (PAT) to authenticate to Collate. This means CAIP agents only have as many permissions as the user, thanks to RBAC integration. * CAIP provides the backend for applications such as Collate AI Chat, Documentation Agent, Tier Agent, and DQ Generation Agent. * Identify the service account used by the Collate server, because it will be needed for the Collate AI Proxy installation. ## Required Access * **Docker images**: Because you already have a Collate installation in place, use the same image-pulling strategy for the Collate AI Proxy Docker images. Reuse the same credentials, whether you are pulling the images from Collate's ECR or from your own container registry. * **Kubernetes access**: Permissions to create and manage resources in your cluster. Verify access, replacing `` with the namespace where Collate is deployed: ```shell theme={null} kubectl auth can-i create deployments -n ``` * **LLM provider credentials**: Credentials for your chosen LLM provider, such as AWS Bedrock, OpenAI, Anthropic, Google, or Ollama. * **AWS Bedrock**: AWS access key ID and secret access key, or an IAM role with Bedrock permissions, plus an active Anthropic model subscription. * **OpenAI**: OpenAI API key. * **Google Gemini**: Google Generative AI API key for the GCP project that will be used by your GKE deployment. ## Hardware Requirements The Collate AI Proxy pod requires at least the following resources: * CPU: `300m` * Memory: `2Gi` * Storage: No persistent storage is required. The service is stateless. Ensure that the Kubernetes node where the pod runs can satisfy these requirements. ## Provider Setup Use the following provider-specific setup guidance before deployment. ### Set Up Amazon Bedrock 1. Sign in to the AWS console. 2. Open Amazon Bedrock in a supported commercial region. 3. Ensure the IAM role or user used for setup has the required AWS Marketplace permissions to enable third-party models. 4. Confirm that your AWS account has a valid payment method configured for AWS Marketplace purchases. 5. Open the model catalog in Amazon Bedrock. 6. Select the model you want to use. 7. For Anthropic models, complete the first-time use case form the first time you enable them in the account. 8. Optional: Review the model license terms and control access with IAM or SCP policies before invoking the model, if your team prefers explicit approval before first use. ### Bedrock Access Notes * Access to foundation models is enabled by default in commercial AWS regions when the required AWS Marketplace permissions are present. * The first invocation of a third-party model can trigger an automatic subscription flow in the background. * During the initial setup period, Bedrock access can take a few minutes to become fully available. * If prerequisites are missing, subsequent calls can fail with `AccessDeniedException`. ### Recommended Model Setup For the Collate AI Proxy (CAIP), use Anthropic chat models through Bedrock and ensure your chosen AWS region supports the model and inference option you plan to use. For semantic search or embeddings, confirm that the embedding model you plan to use is available in the same or intended Bedrock region. ### Create the Azure OpenAI Resource 1. Sign in to the [Azure portal](https://portal.azure.com/). 2. Create a dedicated resource group. 3. Navigate to Azure OpenAI (Microsoft Foundry) Service. 4. Create a new Azure OpenAI Service instance. Create an Azure OpenAI Foundry resource 5. Fill in the resource group name created earlier, the resource name, and the region. 6. Keep the remaining settings as default. 7. Select **Review + create**. 8. Wait for Azure to generate the service. ### Deploy the Models You need two separate deployments: one base model for the Collate AI Proxy (CAIP) and one embedding model for Collate Server. 1. Navigate to the service you created. 2. Select **Go to Foundry Portal**. Azure OpenAI service overview with link to Foundry Portal 3. Under **Shared resources**, select **Deployments**. 4. Select **Deploy model**. 5. Choose **Deploy base model**. Azure OpenAI model deployments page 6. Select a suitable chat model such as `gpt-4o` for the CAIP deployment. 7. Once the deployment is created, capture the `apiKey`, `deploymentName`, `apiVersion`, and `azure_endpoint` or `baseUrl` from the deployment preview. Azure OpenAI deployment preview showing endpoint and deployment details 8. Repeat the model deployment process for the vector embedding model used by Collate Server for semantic search, selecting `text-embedding-3-small` under **Embeddings**. 9. Capture the same set of credentials for the embedding deployment. ### Supported Models All OpenAI models, including `gpt-4o`, are supported and the model list is loaded automatically from the OpenAI API. Collate recommends using `gpt-4o` for the Collate AI Proxy (CAIP). ### Enable the Gemini API 1. Sign in to the Google Cloud console. 2. Select the project used by your GKE cluster. 3. Open **APIs & Services** > **Library**. 4. Search for **Gemini API**. 5. Select **Gemini API**. 6. Select **Enable**. Enable the Gemini API service in the Google Cloud console 7. Wait until the API is enabled for the project. Alternatively, enable the API from the command line: ```shell theme={null} gcloud services enable generativelanguage.googleapis.com --project ``` ### Create and Secure the API Key 1. Navigate to **APIs & Services** > **Credentials**. 2. Create an API key for Google Generative AI. Navigate to credentials for the Gemini API service in Google Cloud Create an API key for Gemini API in Google Cloud 3. Restrict the API key to the Gemini API. 4. Apply any organization-required network or application restrictions. 5. Store the API key in your secret management system, such as an external secrets provider or a Kubernetes Secret in the Collate namespace. Alternatively, create and manage Gemini API keys from Google AI Studio. If the GCP project used by your GKE cluster is not visible in Google AI Studio, import the project first, then create the API key for that project. For more information, see the official [Gemini API key documentation](https://ai.google.dev/gemini-api/docs/api-key). ### Recommended Model Setup For the Collate AI Proxy (CAIP) on GCP, use Gemini chat models with `gemini-2.5-pro` as the primary chat model and `gemini-2.5-flash` as the smaller model variant. For semantic search or embeddings, configure embeddings in the Collate application for now. ## Navigation \< Previous Next >