Lineage
The Lineage API lets you manage data lineage relationships between any two entities in Collate. Lineage captures how data flows from sources to destinations, including column-level mappings and pipeline references.Available Endpoints
Get Lineage
Retrieve the lineage graph for an entity by ID or fully qualified name, with configurable upstream and downstream depth.
Add Lineage
Create a lineage edge between two entities with optional column-level mappings, pipeline references, and SQL queries.
Delete Lineage
Remove a lineage edge between two entities.
Export Lineage
Export the lineage graph for an entity as CSV.
Supported Entity Types
Lineage can be established between any entity types in Collate. Common combinations include:| From | To | Example |
|---|---|---|
| Table | Table | ETL transformations between source and target tables |
| Table | Dashboard | Dashboards consuming data from tables |
| Pipeline | Table | Pipelines producing data into tables |
| Topic | Table | Streaming data from Kafka topics into tables |
| Table | ML Model | Tables used as training data for ML models |
Key Concepts
- Edge: A directional lineage relationship from one entity to another
- Column-level lineage: Mapping specific columns from a source entity to columns in a target entity
- Pipeline reference: Associating a pipeline entity that powers the transformation
- Upstream/Downstream depth: Controls how many hops to traverse when querying the lineage graph