Apache Ranger

• Requirements
• Reverse Metadata
• Metadata Ingestion
• Troubleshooting

​

Requirements

​

Apache Ranger Setup

​

Permissions

• Access to Apache Ranger Admin API endpoints
• Write access to policies and tag definitions
• Write access to tag management
• Read access to service definitions for verification

# Create a service user in Apache Ranger with the following permissions:
# - Write access to tag management
# - Write access to policy management
# - Read access to service definitions

​

Connection Details

​

Reverse Metadata

​

How Reverse Metadata Works

1. Configure Ranger as Sink Service: Set up Apache Ranger as a sink service in your reverse metadata configuration
2. Source Service Integration: When you ingest metadata from source services like Snowflake, Trino, or other databases, Collate can sync this metadata back to Ranger
3. Tag Synchronization: Currently, Collate syncs tag information to Apache Ranger, allowing you to manage security policies based on discovered metadata
4. Policy Management: While Collate syncs tags to Ranger, the communication between Ranger and your specific data sources needs to be configured separately

​

Important Considerations

• Service Name Matching: The service name configured in Apache Ranger must match exactly with the service name in Collate for reverse metadata synchronization to work properly
• Tag Synchronization: Currently, Collate only syncs tag information to Ranger.
• Source-Ranger Communication: You are responsible for configuring the communication between Apache Ranger and your actual data sources. Collate only handles the metadata synchronization to Ranger
• Bidirectional Sync: This is currently a one-way sync from Collate to Ranger

​

Tag Synchronization Details

​

What Gets Created During Reverse Metadata Ingestion

• Ranger Resources: The specific entity (database, schema, table, or column)
• Tags: The tag name and tag value

​

Policy Management

1. Syncing tag metadata from Collate to Ranger
2. Creating tag-to-resource mappings in Ranger
3. Keeping these mappings synchronized as tags change in Collate

​

Supported Tag Levels

• Database level: Tags applied to entire databases
• Schema level: Tags applied to schemas
• Table level: Tags applied to tables
• Column level: Tags applied to individual columns

​

Tag Naming Convention

classification.tag

• A tag named Sensitive under the PII classification in Collate
• Will be synced to Ranger as: PII.Sensitive

​

Complete Workflow Example

1. In Collate: You apply the tag PII.Sensitive to a column customer_email in table users
2. Reverse Metadata Sync: Collate creates a mapping in Ranger linking the resource database.schema.users.customer_email to tag PII.Sensitive
3. In Apache Ranger: You create a policy that applies masking to all resources tagged with PII.Sensitive
4. Result: The policy automatically applies to customer_email and any other resources tagged as PII.Sensitive

​

Metadata Ingestion

​

Step 1: Add New Service

1. Navigate to Settings > Services and select the Security service.
2. Click Add New Service.

​

Step 2: Select a Service and Connector

​

Step 3: Add Service Name and Description

• Enter a unique, descriptive Service Name. Collate identifies services by their service name. Enter a name that distinguishes this deployment from other Ranger services you are ingesting metadata from.
• Optional: Enter a Description for the service.

​

Step 4: Configure Connection Options

​

Select Ingestion Runner

​

Enter Connection Details

• Host and Port: The URL of your Apache Ranger Admin instance (for example, http://ranger-host:6080).
• Username: The username of the Ranger Admin user. The specified user should have access to the Apache Ranger Admin API with privileges to manage policies and tags.
• Password: The password for the Ranger Admin user.

​

Test Connection

1. Click Test Connection to verify the credentials.
2. After the test succeeds, click Save.

​

Step 5: Configure Ingestion Options

​

How Filter Patterns Work

• Include: Add one or more comma-separated values. Each value uses one of the following match types. Collate ingests only assets whose names match at least one rule. Leave blank to include all assets.
• Exclude: Add one or more comma-separated values. Each value uses one of the following match types. Collate skips any asset whose name matches a rule. Leave blank to exclude nothing.

• contains: matches any name containing the value. For example, sales matches my_sales_data and sales_2024.
• starts with: matches names beginning with the value. For example, prod_ matches prod_db and prod_schema.
• ends with: matches names ending with the value. For example, _raw matches events_raw and logs_raw.
• is exactly: matches the exact name only. For example, analytics matches only analytics.
• matches regex: matches names using a regular expression. For example, ^prod_.*_v\d+$ matches prod_events_v1.

​

Step 6: Create & Deploy

​

Troubleshooting