Skip to main content

​
Setup Multiple dbt Projects

Collate supports ingesting metadata from multiple dbt projects in a single workflow — each project can have its own manifest.json, catalog.json, and run_results.json files.
Tip: Multiple dbt project support is available for S3, GCS, and Azure configurations only.
Before you configure the workflow, organize your dbt files so Collate can find them:
  • Place each project’s files in its own directory and name them manifest.json, catalog.json, and run_results.json.
  • If your dbt tests produce multiple run_results.json files, keep them in the same directory as their corresponding manifest.json. Add a unique suffix to each file name — for example: run_results_one.json, run_results_two.json, run_results_three.json.
Collate scans the prefix path you specify and traverses each subfolder to locate the dbt files automatically.

​
How to Configure Multiple dbt Projects (S3 Example)

Follow these steps to set up the dbt workflow for multiple projects using an S3 bucket.

​
Step 1: Organize the dbt Files in Your S3 Bucket

Arrange each dbt project in a separate directory inside your bucket. For example, here’s a bucket named dbt_bucket with three dbt projects:

​
Step 2: Configure the dbt Workflow in Collate

Set the bucket and prefix path so Collate knows where to look.
  • In the dbt Bucket Name field, enter your bucket name — in this example, dbt_bucket.
  • In the dbt Object Prefix field, enter the path to the parent folder containing your projects — in this example, bucket_home/dbt_files/.
To scan the entire bucket, leave the dbt Object Prefix field empty. dbt prefix configuration example

​
Step 3: Run the dbt Workflow

Trigger the workflow and verify all three projects are ingested. After the workflow completes, Collate ingests the dbt metadata from all three projects in a single run.