Custom Tests

Publish results from an external quality tool or implement a validator that Collate runs. Steps 1–4 register a custom test and publish its results through the API. Step 5 describes installing a custom validator for execution through Collate. Before starting, ingest the target table into Collate. Obtain a bearer token with permission to create test definitions, edit tests on the table, and publish results. Replace {access_token} in the examples and replace https://your-company.getcollate.io with your Collate instance URL. The examples use the existing table local_redshift.dev.dbt_jaffle.customers. Replace this fully qualified name (FQN) throughout with your table’s FQN. The workflow has five steps. Steps 2, 4, and 5 are optional, as described below:

Create a Test Definition
Create a Test Suite
Create a Test Case
Write Test Case Results
Make the Test Available Through the Collate UI

Step 1: Create a Test Definition

Send POST /api/v1/dataQuality/testDefinitions to register a definition. This example declares a string parameter named colName for a test whose results are calculated externally.

curl --request POST 'https://your-company.getcollate.io/api/v1/dataQuality/testDefinitions' \
--header 'Authorization: Bearer {access_token}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "description": "A demo custom test",
    "entityType": "TABLE",
    "name": "demo_test_definition",
    "testPlatforms": ["Other"],
    "parameterDefinition": [{
        "name": "colName",
        "dataType": "STRING",
        "required": true
    }]
}'

Keep the fullyQualifiedName from the response. This example returns demo_test_definition. Test-case creation accepts this name as a string, rather than an ID or an entity-reference object. To make the definition available for execution through Collate, include OpenMetadata in testPlatforms and install a matching validator as described in Step 5. Registering a definition alone doesn’t implement or run the test.

Step 2: Create a Test Suite

Optional: Create a suite explicitly before creating a test case. A basic test suite holds the tests for a table. When you create a test case in Step 3, the server reuses the table’s basic suite or creates one automatically. Skip this step unless you want to create the suite explicitly. To create it explicitly, send POST /api/v1/dataQuality/testSuites/basic with the existing table’s FQN in basicEntityReference. Reuse the suite if the table already has one.

curl --request POST 'https://your-company.getcollate.io/api/v1/dataQuality/testSuites/basic' \
--header 'Authorization: Bearer {access_token}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "name": "local_redshift.dev.dbt_jaffle.customers.testSuite",
    "description": "Custom tests for the customers table",
    "basicEntityReference": "local_redshift.dev.dbt_jaffle.customers"
}'

POST /api/v1/dataQuality/testSuites creates a logical grouping instead. Omit executable and basic from the request. The endpoint determines the suite type. Use basicEntityReference instead of the deprecated executableEntityReference field. For more information, see Create a Test Suite.

Step 3: Create a Test Case

Send POST /api/v1/dataQuality/testCases with the test definition’s FQN as a string and an entityLink identifying the table or column. For a table, use <#E::table::tableFQN>. For a column, use <#E::table::tableFQN::columns::columnName>. Include the opening and closing angle brackets. Omit testSuite from the request. The server associates the case with the table’s basic suite using entityLink. Parameter names must match the definition, and parameter values must be strings.

curl --request POST 'https://your-company.getcollate.io/api/v1/dataQuality/testCases' \
--header 'Authorization: Bearer {access_token}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "entityLink": "<#E::table::local_redshift.dev.dbt_jaffle.customers>",
    "name": "custom_test_Case",
    "testDefinition": "demo_test_definition",
    "parameterValues": [{
        "name": "colName",
        "value": "id"
    }]
}'

Keep the returned fullyQualifiedName for Step 4. For this example, it’s local_redshift.dev.dbt_jaffle.customers.custom_test_Case.

Step 4: Write Test Case Results

Optional: Publish results yourself when you run the test outside Collate. After running your external test, send POST /api/v1/dataQuality/testCases/testCaseResults/{fqn} with its result. Use the test case’s returned FQN, URL-encoded when necessary, in the path. The required fields are timestamp, testCaseStatus, and testResultValue. The timestamp is Unix epoch time in milliseconds. Replace the example timestamp with the actual execution time.

curl --request POST 'https://your-company.getcollate.io/api/v1/dataQuality/testCases/testCaseResults/local_redshift.dev.dbt_jaffle.customers.custom_test_Case' \
--header 'Authorization: Bearer {access_token}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "result": "All 10 checked rows passed the custom validation",
    "testCaseStatus": "Success",
    "timestamp": 1769982800000,
    "testResultValue": [{
        "name": "rowsChecked",
        "value": "10"
    }]
}'

View the result on the table’s test case. For more information, see Test Case Results.

Step 5: Make the Test Available Through the Collate UI

Optional: To run the test through the Collate user interface (UI), implement a validator in the metadata.data_quality namespace.

Create Your Namespace Package Create a Python package for your validation logic with this minimum structure:
metadata/ setup.py
For SQLAlchemy sources, place your validation file in the directory for its entity type:
- Table tests: metadata/data_quality/validations/table/sqlalchemy/<yourTest>.py
- Column tests: metadata/data_quality/validations/column/sqlalchemy/<yourTest>.py
<yourTest> should match the name of your test definition in Step 1. Add an __init__.py file in every package folder with this line:
__path__ = __import__('pkgutil').extend_path(__path__, __name__)
Create Your Test Class In <yourTest>.py, create a class named <YourTest>Validator that inherits from BaseTestValidator. Inherit from SQAValidatorMixin if you need its helper methods. Implement run_validation to return a TestCaseResult object. The following illustrative skeleton omits imports and validation logic. Add these in your implementation before running it.
class ColumnEntropyToBeBetweenValidator(BaseTestValidator): """Implements custom test validator for Collate. Args: BaseTestValidator (_type_): inherits from BaseTestValidator """ def run_validation(self) -> TestCaseResult: """Run test validation"""
Install Your Package After implementing the validator, run this command from your package directory in the environment that runs the Collate Python SDK:
pip install .

​Custom Tests

​Step 1: Create a Test Definition

​Step 2: Create a Test Suite

​Step 3: Create a Test Case

​Step 4: Write Test Case Results

​Step 5: Make the Test Available Through the Collate UI