> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getcollate.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Custom Tests

> Build and manage custom data quality tests to validate schema, values, and distributions during ingestion.

## Custom Tests

Collate's built-in tests cover most use cases, but sometimes you need something more specific. You can bring in results from your own test suite or define entirely new tests that run inside Collate — all through the API and the Python SDK.

The setup involves five steps:

1. [Create a Test Definition](#step-1-create-a-test-definition)
2. [Create a Test Suite](#step-2-create-a-test-suite)
3. [Create a Test Case](#step-3-create-a-test-case)
4. [Write Test Case Results](#step-4-write-test-case-results)
5. [(Optional) Make the test available in the Collate UI](#step-5-optional-make-your-test-available-in-the-collate-ui)

### Step 1: Create a Test Definition

A Test Definition tells Collate what your test is: its name, what entity type it applies to (table or column), and which platforms run it.

Send a `POST` request to `/api/v1/dataQuality/testDefinition` with at least the following fields:

```json theme={null}
{
    "description": "<your test definition description>",
    "entityType": "<TABLE or COLUMN>",
    "name": "<your_test_name>",
    "testPlatforms": ["<any of Collate,GreatExpectations, dbt, Deequ, Soda, Other>"],
    "parameterDefinition": [
      {
        "name": "<name>"
      },
      {
        "name": "<name>"
      }
    ]
}
```

Here is a complete cURL example:

```bash theme={null}
curl --request POST 'http://localhost:8585/api/v1/dataQuality/testDefinitions' \
--header 'Content-Type: application/json' \
--data-raw '{
    "description": "A demo custom test",
    "entityType": "TABLE",
    "name": "demo_test_definition",
    "testPlatforms": ["Soda", "dbt"],
    "parameterDefinition": [{
        "name": "ColumnOne"
    }]
}'
```

**Save the universally unique identifier (UUID) from the response** — you'll need it to create the Test Case.

**Heads up**: To make this test definition available in the Collate UI, include `Collate` in `testPlatforms`. That requires extra setup covered in [Step 5](#step-5-optional-make-your-test-available-in-the-collate-ui). If you only want to build a UI-executable test, go to [Step 5](#step-5-optional-make-your-test-available-in-the-collate-ui).

### Step 2: Create a Test Suite

A Test Suite groups Test Cases together for scheduling and ownership. You can create a new one or reuse an existing suite.

Send a `POST` request to `/api/v1/dataQuality/testSuites/executable`:

```json theme={null}
{
  "name": "<test_suite_name>",
  "description": "<test suite description>",
  "executableEntityReference": "<entityFQN>"
}
```

Here is a complete cURL example:

```bash theme={null}
curl --request POST 'http://localhost:8585/api/v1/dataQuality/testSuites/executable' \
--header 'Content-Type: application/json' \
--data-raw '{
  "name": "<test_suite_name>",
  "description": "<test suite description>",
  "executableEntityReference": "<entityFQN>"
}'
```

**Save the UUID from the response** — you'll need it in the next step.

### Step 3: Create a Test Case

A Test Case is a specific instance of your Test Definition applied to a table or column. It uses the fully qualified name (FQN) to reference the test definition and suite.

Send a `POST` request to `/api/v1/dataQuality/testCases`:

```json theme={null}
{
    "entityLink": "<#E::table::fqn> or <#E::table::fqn::columns::column name>",
    "name": "<test_case_name>",
    "testDefinition": {
        "FQN": "<test definition FQN>",
        "type": "testDefinition"
    },
    "testSuite": {
        "FQN": "<test suite FQN>",
        "type": "testSuite"
    }
}
```

**Important**: Include the opening and closing `<>` in the `entityLink` value.

Here is a complete cURL example:

```bash theme={null}
curl --request POST 'http://localhost:8585/api/v1/dataQuality/testCases' \
--header 'Content-Type: application/json' \
--data-raw '{
    "entityLink": "<#E::table::local_redshift.dev.dbt_jaffle.customers>",
    "name": "custom_test_Case",
    "testDefinition": {
        "id": "1f3ce6f5-67be-45db-8314-2ee42d73239f",
        "type": "testDefinition"
    },
    "testSuite": {
        "id": "3192ed9b-5907-475d-a623-1b3a1ef4a2f6",
        "type": "testSuite"
    },
    "parameterValues": [
        {
            "name": "colName",
            "value": 10
        }
    ]
}'
```

Save the UUID from the response

### Step 4: Write Test Case Results

<Note>
  **Note**: This step is optional if you're running the test through the Collate UI.
</Note>

After your Test Case exists, push results to it with a `PUT` request to `/api/v1/dataQuality/testCases/{test FQN}/testCaseResult`:

```json theme={null}
{
    "result": "<result message>",
    "testCaseStatus": "<Success or Failed or Aborted>",
    "timestamp": <Unix timestamp in milliseconds>,
    "testResultValue": [
      {
        "value": "<value>"
      }
    ]
}
```

Here is a complete cURL example:

```bash theme={null}
curl --location --request PUT 'http://localhost:8585/api/v1/dataQuality/testCases/local_redshift.dev.dbt_jaffle.customers.custom_test_Case/testCaseResult' \
--header 'Content-Type: application/json' \
--data-raw '{
    "result": "found 1 value, expected n",
    "testCaseStatus": "Success",
    "timestamp": 1662129151000,
    "testResultValue": [{
        "value": "10"
    }]
}'
```

Your test now appears in the Test Suite and on the table entity page.

### Step 5: (Optional) Make Your Test Available in the Collate UI

To make your custom test available in the Collate UI — so anyone can run it without touching the API — wire it up using the Collate `data_quality` namespace submodule.

1. **Create your namespace package**

   Start by creating a Python package that holds your test logic. At minimum, your package needs this structure:

   ```
   metadata/
   setup.py
   ```

   Place your test file in the right location based on the entity type:

   * Table tests: `metadata/data_quality/validations/table/sqlalchemy/<yourTest>.py`
   * Column tests: `metadata/data_quality/validations/column/sqlalchemy/<yourTest>.py`

   The filename (`<yourTest>`) must match the test name you used in Step 1.

   **Important:** Add an `__init__.py` file to every folder with this line:

   ```python theme={null}
   __path__ = __import__('pkgutil').extend_path(__path__, __name__)
   ```

2. **Create your test class**

   In your `<yourTest>.py` file, create a class named `<YourTest>Validator` that inherits from `BaseTestValidator`. Optionally inherit from `SQAValidatorMixin` too — it gives you extra helper methods out of the box. Implement the `run_validation` method, which must return a `TestCaseResult` object.

   See a full working example in the [OpenMetadata validator source tree](https://github.com/open-metadata/OpenMetadata/tree/main/ingestion/src/metadata/data_quality/validations) — it implements an entropy test end to end.

   ```python theme={null}
   class ColumnEntropyToBeBetweenValidator(BaseTestValidator):
       """Implements custom test validator for Collate."""

       def run_validation(self) -> TestCaseResult:
           """Run test validation"""
   ```

3. **Install your package**

   Once your package is ready, install it in the same environment where the Collate Python SDK is installed:

   ```
   pip install .
   ```

   <img src="https://mintcdn.com/collatedocs/hzvCWOBUMdmV543T/public/images/features/ingestion/workflows/data-quality/custom-test-definition.png?fit=max&auto=format&n=hzvCWOBUMdmV543T&q=85&s=bf46995d95ec4c85098dea7451644f88" alt="Custom test definition created in Collate UI" width="1864" height="2515" data-path="public/images/features/ingestion/workflows/data-quality/custom-test-definition.png" />

   <img src="https://mintcdn.com/collatedocs/hzvCWOBUMdmV543T/public/images/features/ingestion/workflows/data-quality/custom-test-result.png?fit=max&auto=format&n=hzvCWOBUMdmV543T&q=85&s=96ae5018eb735c761b20f6b426a587d3" alt="Custom test case result displayed in Collate UI" width="4024" height="1911" data-path="public/images/features/ingestion/workflows/data-quality/custom-test-result.png" />