Skip to main content

dbt Cloud API Configuration

This guide shows you how to integrate dbt Cloud with Collate using the dbt Cloud API.
This guide is ONLY for dbt Cloud users. If you’re using dbt Core, see dbt Core Configuration for storage setup options.

Prerequisites Checklist

RequirementDetailsHow to Verify
dbt Cloud AccountActive subscriptionCan log in to cloud.getdbt.com
dbt Cloud JobsAt least one successful job runCheck job run history
API AccessAccount Viewer permission minimumCan create API tokens
Database ServiceData warehouse already ingestedCheck Settings → Services

Step 1: Get dbt Cloud API Token

1.1 Create API Token

  1. Log in to dbt Cloud (cloud.getdbt.com)
  2. Click your profile icon (top right)
  3. Go to Account Settings
  4. Click API Access in the left sidebar
  5. Click + Create Token
  6. Configure the token:
    • Description: Collate Integration
    • Permission: Account Viewer
  7. Click Save
  8. Copy the token immediately - you won’t see it again!
Expected token format:
dbt_cloud_abc123def456ghi789jkl...
Save this token securely. You’ll need it for Collate configuration.

1.2 Find Your Account ID

Method A: From URL (Easiest) When logged into dbt Cloud, look at the browser URL:
https://cloud.getdbt.com/accounts/12345/projects/...
                                 ^^^^^
                         This is your Account ID
Method B: Via API
export DBT_CLOUD_TOKEN="your-token-here"

curl -H "Authorization: Token ${DBT_CLOUD_TOKEN}" \
     https://cloud.getdbt.com/api/v2/accounts/ | jq '.data[].id'

1.3 Identify Your dbt Cloud Region

dbt Cloud has regional deployments:
RegionURLUse If
North Americahttps://cloud.getdbt.comDefault (most common)
EMEAhttps://emea.dbt.comEuropean accounts
APAChttps://au.dbt.comAsia-Pacific accounts
How to check: Look at your dbt Cloud login URL.

1.4 (Optional) Get Project/Job IDs

By default, Collate will use the most recent successful run from any project/job. To filter to a specific project or job, you’ll need their IDs. Get IDs from dbt Cloud UI:
  • Account ID: In URL: https://cloud.getdbt.com/#/accounts/{ACCOUNT_ID}/...
  • Project ID: Go to Account Settings → Projects, click a project, ID is in the URL
  • Job ID: Go to Deploy → Jobs, click a job, ID is in the URL
Or retrieve via API: List Projects:
export ACCOUNT_ID="12345"

curl -H "Authorization: Token ${DBT_CLOUD_TOKEN}" \
     https://cloud.getdbt.com/api/v2/accounts/${ACCOUNT_ID}/projects/ | \
     jq '.data[] | {id, name}'
Example output:
{
  "id": 98765,
  "name": "analytics"
}
List Jobs:
curl -H "Authorization: Token ${DBT_CLOUD_TOKEN}" \
     https://cloud.getdbt.com/api/v2/accounts/${ACCOUNT_ID}/jobs/ | \
     jq '.data[] | {id, name, project_id}'
Example output:
{
  "id": 54321,
  "name": "Daily Production Run",
  "project_id": 98765
}

Step 2: Configure Collate

Configuration

  1. Go to Settings → Services → Database Services
  2. Click on your database service (e.g., “production-snowflake”)
    • Important: Must match the database dbt connects to
  3. Go to the Ingestion tab
  4. Click Add Ingestion
  5. Select dbt from the dropdown
Configure dbt Source (dbt Cloud):
FieldValueNotes
dbt Configuration Sourcedbt CloudSelect from dropdown
dbt Cloud Auth Tokendbt_cloud_abc123...Your API token from Step 1.1
dbt Cloud Account ID12345Your account ID from Step 1.2
dbt Cloud URLhttps://cloud.getdbt.comOr your regional URL
dbt Cloud Project ID(optional)Leave empty to use latest run from any project
dbt Cloud Job ID(optional)Leave empty to use latest run from any job
How filtering works:
  • Both empty: Uses most recent successful run from any job in the account
  • Project ID provided: Uses most recent run from that specific project
  • Both provided: Uses most recent run from that specific job
Configure dbt Options:
FieldRecommended Value
Update DescriptionsEnabled
Update OwnersEnabled
Include TagsEnabled
Classification NamedbtTags
Match your schedule to dbt Cloud job frequency. If jobs run every 6 hours, schedule Collate ingestion every 6 hours.
Test & Deploy:
  1. Click Test Connection
    • Should show: ✓ API connectivity, ✓ Account access, ✓ Recent run found
  2. If successful, click Deploy
  3. Click Run to trigger immediately

Verification

Verification Checklist

CheckHow to VerifyExpected Result
dbt Cloud run existsCheck dbt Cloud job historyRecent successful run visible
Ingestion completedCollate UI → Service → Ingestion tabGreen status, no errors
Lineage appearsClick on a dbt model → Lineage tabUpstream/downstream connections
Descriptions syncedClick on a table → Schema tabdbt descriptions visible
Test results visibleClick on a table → Data Quality tabdbt test results shown

Best Practices

Ensure Jobs Generate All Artifacts

Your dbt Cloud job must include these commands:
dbt run           # Generates manifest.json
dbt test          # Generates run_results.json with test results
dbt docs generate # Generates catalog.json with column descriptions
How to verify:
  1. Go to dbt Cloud → Jobs
  2. Click on your job
  3. Go to Settings → Commands
  4. Confirm all three commands are present

Schedule Alignment

dbt Cloud Job FrequencyRecommended Collate Schedule
Hourly0 * * * * (Every hour)
Every 6 hours0 */6 * * * (Every 6 hours)
Daily (6 AM)0 7 * * * (Daily at 7 AM)
On-demand only0 */3 * * * (Every 3 hours)

API Rate Limits

dbt Cloud API has rate limits:
  • 100 requests per minute per API token
Collate makes ~3-5 API calls per ingestion. Don’t schedule more frequently than every 15 minutes.

Troubleshooting

IssueSymptomCauseSolution
Invalid Token”401 Unauthorized”Token expired or wrongRegenerate API token in dbt Cloud
No Runs Found”No recent successful runs”No completed jobsRun a dbt Cloud job and wait for completion
Wrong Account”403 Forbidden”Account ID incorrectVerify Account ID from URL or API
Missing catalog.jsonNo column descriptionsJob didn’t run dbt docs generateAdd docs generation to job commands
Wrong Database ServiceNo lineage appearsService name mismatchEnsure Collate service name matches dbt target
Stale dataOld metadata showingIngestion not runningCheck ingestion schedule and logs

Debug Commands

# Test API connectivity
curl -H "Authorization: Token ${DBT_CLOUD_TOKEN}" \
     https://cloud.getdbt.com/api/v2/accounts/${ACCOUNT_ID}/ | jq '.'

# Get most recent successful run (what Collate will use)
curl -H "Authorization: Token ${DBT_CLOUD_TOKEN}" \
     "https://cloud.getdbt.com/api/v2/accounts/${ACCOUNT_ID}/runs/?status=10&order_by=-finished_at&limit=1" | \
     jq '.data[0] | {id, job_id, finished_at, status_humanized}'

# Download manifest from run
curl -H "Authorization: Token ${DBT_CLOUD_TOKEN}" \
     "https://cloud.getdbt.com/api/v2/accounts/${ACCOUNT_ID}/runs/${RUN_ID}/artifacts/manifest.json" | \
     jq '.metadata.dbt_version'

Next Steps

Questions? See the main dbt Overview or visit our documentation.