> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getcollate.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Auto PII Tagging

> Automatically tag personally identifiable information (PII) in your datasets to support data classification, compliance, and privacy enforcement.

# Auto PII Tagging

Auto PII tagging for Sensitive/NonSensitive at the column level is performed based on the two approaches described below.

## Tagging logic

1. **Column Name Scanner**: We validate the column names of the table against a set of regex rules that help us identify
   common English patterns to identify email addresses, SSN, bank accounts, etc.
2. **Entity Recognition**: The workflow samples rows directly from the source and validates them against an Entity
   Recognition engine that identifies sensitive information from a list of [supported entities](https://microsoft.github.io/presidio/supported_entities/).
   The `confidence` parameter lets you tune the minimum score required to tag a column as `PII.Sensitive`.
   Enabling **Store Sample Data** saves those rows in OpenMetadata, but is not required for classification to run.

Note that if a column is already tagged as `PII`, we will ignore its execution.

## Auto-classification vs Governance Automations

Both can apply tags to columns, but they work differently:

|                      | Auto-classification                                                                                                                       | Governance Automations                                                                       |
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| How it works         | Scans sample data content and column names using recognizers                                                                              | Applies tags to entities matching rules you define (name, service, schema, owner, and so on) |
| Requires sample data | The workflow samples rows on-the-fly from the source — no pre-stored sample data needed. **Store Sample Data** controls persistence only. | No — operates on metadata stored in the search index                                         |
| Best for             | Discovering unknown sensitive data automatically across large datasets                                                                    | Tagging columns you already know need a specific tag                                         |
| Limitation           | Won't tag if sampled values are masked, anonymized, or don't meet the confidence threshold                                                | Manual — you must define a rule for each case                                                |

Use **Governance Automations** when:

* The source cannot be reached or sampled during the profiler run
* Sampled values are masked, anonymized, or don't match the expected pattern
* You want to tag all columns matching a name pattern (for example, all columns named `person_id` or `email`)

## Troubleshooting

### Recognizer ran but tag was not applied

If the auto-classification logs show a recognizer attempted to classify a column but no tag appears:

* **Sampled values are masked or anonymized** — the content recognizer found no match above the confidence threshold. Switch the recognizer `Target` to **Column Name** to tag based on the column name alone, regardless of data content.
* **Confidence score too low** — the recognizer ran but scored below the threshold. Lower the confidence threshold or add context to boost the score.
* **Column already tagged as PII** — columns with an existing `PII` tag are skipped. Check the column's existing tags.

<Tip>
  In Collate, the auto-classification feature primarily applies the PII classification, tagging data as either Sensitive or Non-Sensitive. The General classification, which includes tags like Address, Name, etc., is not available in Collate. This functionality is present in the Collate and is expected to be included in the open-source release starting from version 1.7.1.
</Tip>

### SSL: CERTIFICATE\_VERIFY\_FAILED

If you see an error similar to:

```
Unexpected error while processing sample data for auto pii tagging - HTTPSConnectionPool(host='raw.githubusercontent.com', port=443):
Max retries exceeded with url: /explosion/spacy-models/master/compatibility.json
(Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to
get local issuer certificate (_ssl.c:1129)')))
```

This is a scenario that we identified on some corporate Windows laptops. The bottom-line here is that the profiler
is trying to download the Entity Recognition model but having certificate issues when trying the request.

A solution here is to manually download the model on the ingestion container / Airflow host by running:

```
pip --trusted-host github.com --trusted-host objects.githubusercontent.com install https://github.com/explosion/spacy-models/releases/download/en_core_web_md-3.5.0/en_core_web_md-3.5.0.tar.gz
```

If using Docker, you might want to customize the `Collate-ingestion` image to have this command run there by default.