Documentation Index
Fetch the complete documentation index at: https://docs.getcollate.io/llms.txt
Use this file to discover all available pages before exploring further.
Requirements
Before deploying the Hybrid Runner, ensure your environment meets the following common prerequisites. These requirements cover the minimum hardware, software, and secrets management setup needed for a successful installation.Hardware
A Kubernetes cluster with at least two worker nodes is required. Each node must meet the following minimum specifications:- 2 vCPUs (supports x86 and ARM64 architectures)
- 8 GiB memory
- 64 GiB storage
Recommended Instance Types
The following instance types are verified to meet these requirements across major cloud providers:| Cloud provider | Recommended instance type |
|---|---|
| GKE | t2a-standard-2 or t2d-standard-2 |
| EKS | m6a.large |
| AKS | b2as v2 |
Software Requirements
The following tools must be installed before deploying the Hybrid Runner:- Kubernetes 1.28 or later
helm– installed locallykubectl– installed and configured to target your cluster- Argo Workflows 3.7 – installed when
installArgoWorkflows: trueis set in your Helm values
Spot and Preemptible Node Support
Ingestion workloads can run on Spot VMs (GKE, Azure) or Spot Instances (EKS) to reduce costs. At least one node must remain running at all times to keep the Hybrid Runner pod and Argo Workflows controller available. To schedule Collate workloads on dedicated nodes, use Kubernetes taints and tolerations. The Hybrid Runner supports tolerations via custom Helm values. See Advanced Configuration.Secrets Store
Why the Hybrid Runner Is Designed This Way
The secrets store requirement is a deliberate security design. The Hybrid Runner ensures Collate never sees, stores, or transmits your connector credentials. Here’s how it works:- Credentials live exclusively in your own infrastructure — in your Kubernetes cluster or your cloud-native secrets manager.
- When an ingestion job runs, the ingestion pod resolves the credential directly from your secrets store at runtime, inside your cluster.
- Collate stores only a secret reference path (for example,
secret:my-db-password) – never the credential value. - The Collate server passes the reference to the Runner over the WebSocket connection. The Runner resolves the actual value locally, and it never leaves your environment.
- Your database passwords, API keys, and tokens stay entirely within your perimeter.
- A breach of the Collate platform can’t expose your connector credentials.
- You keep full control over credential rotation, access policies, and audit trails through your own secrets infrastructure.
Supported Secrets Store Options
| Option | Description | Best for |
|---|---|---|
| Kubernetes Secrets (default) | Native Kubernetes secrets; the Helm chart installs the required RBAC automatically | Most users – no additional setup needed |
| AWS Secrets Manager | Cloud-managed secrets via IAM (Identity and Access Management) and IRSA (IAM Roles for Service Accounts) or EKS Pod Identity | AWS EKS (Elastic Kubernetes Service) deployments |
| GCP Secret Manager | Cloud-managed secrets via Workload Identity | GKE (Google Kubernetes Engine) deployments |
| Azure Key Vault | Cloud-managed secrets via Workload Identity and Managed Identity | AKS (Azure Kubernetes Service) deployments |
Obtain Collate JSON Web Token (JWT)
The Hybrid Runner authenticates to Collate using a JWT (JSON Web Token). Get your token using one of these two methods.Option 1 – Use an Existing Bot Token
Collate includes a pre-configuredingestion-bot you can use straight away.
- Log in to your Collate platform.
- Navigate to Settings > Bots.
- Select the
ingestion-bot. - Copy the JWT token and save it for later use.
Option 2 – Create a Custom Bot
For production use, create a dedicated bot with specific permissions.- Go to Settings > Bots.
- Select Add Bot.
- Enter a Name and Description.
- Assign the appropriate roles – typically
DefaultBotPolicyandIngestion Bot Policy. - Copy the generated JWT token and save it for later use.