Skip to main content

​
Pagination

The Collate API uses cursor-based pagination for list endpoints. This ensures consistent results even when data changes between requests.

How It Works

{
  "data": [
    { "id": "...", "name": "table1", ... },
    { "id": "...", "name": "table2", ... }
  ],
  "paging": {
    "total": 150,
    "after": "eyJsYXN0SWQiOiIxMjM0NTY3ODkwIn0="
  }
}

​
Pagination Parameters

ParameterTypeDefaultDescription
limitinteger10Number of results per page (max 1000)
beforestring-Cursor for previous page
afterstring-Cursor for next page

​
Response Fields

The paging object contains:
FieldTypeDescription
totalintegerTotal count of matching resources
beforestringCursor for the previous page (if available)
afterstringCursor for the next page (if available)

​
Examples

​
Basic Pagination

Basic Pagination
from metadata.ingestion.ometa.ometa_api import OpenMetadata
from metadata.generated.schema.entity.data.table import Table

metadata = OpenMetadata(config)

# Get first page
first_page = metadata.list_entities(
    entity=Table,
    limit=20
)

print(f"Total tables: {first_page.paging.total}")
for table in first_page.entities:
    print(table.fullyQualifiedName)

# Get next page using after cursor
if first_page.paging.after:
    next_page = metadata.list_entities(
        entity=Table,
        limit=20,
        after=first_page.paging.after
    )
    for table in next_page.entities:
        print(table.fullyQualifiedName)

​
Iterating Through All Results

Iterating Results
from metadata.ingestion.ometa.ometa_api import OpenMetadata
from metadata.generated.schema.entity.data.table import Table

metadata = OpenMetadata(config)

# Iterate through all tables
for table in metadata.list_all_entities(entity=Table, limit=100):
    print(table.fullyQualifiedName)
    # Process each table...

​
Filtering with Pagination

Combine pagination with filters for efficient data retrieval:
Filtering
from metadata.ingestion.ometa.ometa_api import OpenMetadata
from metadata.generated.schema.entity.data.table import Table

metadata = OpenMetadata(config)

# List tables from a specific database
tables = metadata.list_entities(
    entity=Table,
    params={"database": "prod.analytics"},
    limit=50
)

for table in tables.entities:
    print(table.fullyQualifiedName)

​
Include Fields

Control which fields are returned in the response using the fields parameter: 
# Request specific fields
curl -X GET "https://your-company.getcollate.io/api/v1/tables?fields=owner,tags,columns&limit=20" \
  -H "Authorization: Bearer $TOKEN"
Common field options for tables:
  • owner - Include owner information
  • tags - Include tags and classifications
  • columns - Include column definitions
  • followers - Include followers
  • tableConstraints - Include constraints
  • usageSummary - Include usage statistics

​
Best Practices

1

Use reasonable page sizes

Start with limit=50-100. Larger pages reduce API calls but increase memory usage.
2

Don't skip pages

Always use cursors sequentially. Don’t try to construct cursor values manually.
3

Handle empty results

Check if data array is empty or after cursor is null to detect end of results.
4

Request only needed fields

Use the fields parameter to reduce response size and improve performance.
5

Implement retry logic

Handle transient errors gracefully when paginating through large datasets.
For finding specific resources, consider using the Search API instead of paginating through all results: 
# Search is faster for finding specific resources
curl -X GET "https://your-company.getcollate.io/api/v1/search/query?q=customers&index=table_search_index" \
  -H "Authorization: Bearer $TOKEN"

Search API

Learn about searching and filtering metadata