Skip to main content

Lineage

Lineage itself is mostly additive in 2.0 — new FQN-based edge APIs, batch hydration, richer change events. The two things that actually need attention are a renderer swap that breaks forks, and a new time-window filter whose traversal semantics are easy to misread as a clean point-in-time snapshot.

Knowledge Graph renderer replaced

Breaking for forks that import the graph components. The AntV-G6 based knowledge graph is removed and replaced by an RDF-backed 3D Knowledge Graph tab with an Ontology mode and full-viewport expand. The components/KnowledgeGraph/** modules and NodeContextMenu are deleted; replacements live under components/OntologyExplorer/**.
The 3D graph reads from the RDF store — if RDF is disabled, the tab has no data. RDF inference also now defaults to off. Set RDF_INFERENCE_ENABLED=true to restore 1.13 behaviour for SPARQL queries relying on inferred triples.
The wider lineage UI also changes (behavioural, affects users and fork maintainers):

Time-window lineage has surprising traversal semantics

New capability, read before using it. The search-lineage and entity-count-lineage requests gain startTime/endTime (epoch milliseconds), matched by range overlap on edge createdAt/updatedAt. The window is applied as a hard prune during graph traversal, not as a post-filter. An out-of-window edge severs discovery of everything reachable only through it, in both directions — a node is shown only if it’s reachable from the root through edges that are all in-window. Two consequences worth planning for:
  1. A single stale edge hides a whole subtree. If the edge from A → B falls outside the window, nothing downstream of B is returned even if those edges are in-window.
  2. Legacy edges always match. Edges with no timestamps match any window for backwards compatibility, so a window query over a mix of legacy and temporal data returns all legacy edges plus the in-window temporal ones — don’t read the result as a clean point-in-time snapshot until your graph is fully timestamped.
Setting endTime alone gives as-of semantics: edges that existed on or before that instant.

Lineage change events are more specific

Additive, but affects webhook and event-subscription consumers that filtered broadly. changeEventType adds entityLineageAdded, entityLineageDeleted and entityLineageUpdated — lineage mutations previously surfaced only as generic entity updates, or not at all.
Subscriptions that filtered on entityUpdated to catch lineage changes might now see those changes arrive under the new, more specific types instead.

Batch hydration replaces per-node fetches

Additive. POST /v1/lineage/hydrate accepts 1–200 entities identified by type and id only, grouping hydrated entities by entity type in the response. Graph UIs and lineage exporters that fan out one entity GET per node should switch to a single hydrate call per page of nodes. A new source-scoped delete (DELETE /v1/lineage/source/name/{entityType}/{entityFQN}/type/{lineageSource}) also lets you re-run one lineage source (dbt, OpenLineage, manual) without wiping edges contributed by other sources.

Connector lineage gains new sources

Additive. dbt Cloud gains column-level lineage from compiled SQL, dbt gains semantic-layer metric ingestion with column-level lineage, and Kafka Connect creates lineage for EventRouter and RegexRouter routed topics (see also Ingestion & Connectors for the Snowflake access-history lineage path).

Full detail

For the full narrative walkthrough of Lineage in Collate 2.0 — including screenshots and step-by-step context — see Release 2.0: Lineage.