SSL Troubleshooting

In this section we comment common issues that the user can face when enabling SSL in OpenMetadata.

Bot using JWT as authentication mechanism

After enabling SSL on the OM server, we have to update also the public keys URL for the validation of the JWT tokens by updating to the secured URL: https://{server_domain}:{port}/api/v1/system/config/jwks. In case we are using a self-signed certificate, it will fail with the error below:

To avoid this error, you must import your public certificate into the Java Keystore of the OM server. If your OM deployment is done with Docker or Kubernetes, you must copy the cert into the openmetadata_server container or pod. After that, you can proceed with the following steps from your terminal:

Go to your $JAVA_HOME/lib/security where the cacerts keystore is located.
Run the following command once in the directory:

keytool -import -trustcacerts -keystore cacerts -storepass changeit -noprompt -alias localhost -file /path/to/public.cert

After that, you can restart the server, and the error 500 will disappear.

Deploying workflows in Airflow

One common issue after enabling SSL with a self-signed certificate is that our workflows in Airflow will fail or will not be deployed. We can notice it because the following error will be shown in the UI when deploying or re-deploying:

This can be solved in two different ways:

1. Validate the certificate using the public certificate (recommended):

We specify which public certificate must be used to validate the OM server connection.

Copy the public certificate into our Airflow instance.
Update the configuration of our OM server so that each time a workflow is deployed, we send the new configuration.

In docker:

PIPELINE_SERVICE_CLIENT_VERIFY_SSL=validate
PIPELINE_SERVICE_CLIENT_SSL_CERT_PATH=/path/to/certificate/in/airflow

In bare metal:

Edit the conf/openmetadata.yaml file:

pipelineServiceClientConfiguration:
  verifySSL: "validate"
  sslConfig:
    certificatePath: "/path/to/certificate/in/airflow"

In K8s:

We have to update in the values.yaml file with:

openmetadata:
  config:
    pipelineServiceClientConfig:
      verifySsl: "validate"
      sslCertificatePath: "/path/to/certificate/in/airflow"

2. Ignore the certification validation (not recommended for production):

When doing any call to the secured OM server, the certificate validation will be ignored.

In docker:

PIPELINE_SERVICE_CLIENT_VERIFY_SSL=ignore

In bare metal:

Edit the conf/openmetadata.yaml file:

pipelineServiceClientConfiguration:
  verifySSL: "ignore"

In K8s:

We have to update in the values.yaml file with:

openmetadata:
  config:
    pipelineServiceClientConfig:
      verifySsl: "ignore"

Once one of the configurations is set, we can restart our OM server and deploy or redeploy without any issues.

Ingesting from CLI

Similar to what happens when deploying workflows in Airflow, we have to update our workflow config file with one of these options:

To validate our certificate:

workflowConfig:
  openMetadataServerConfig:
    verifySSL: validate
    sslConfig:
      caCertificate: /local/path/to/certificate

To ignore certificate validation:

workflowConfig:
  openMetadataServerConfig:
    verifySSL: ignore

Demo of SSL enabled with an SSO and JWT token configured

For guidance on combining JWT tokens, SSO configuration, and SSL, see the SSO deployment guides and SSL configuration guide.

Providing a single keystore that has all the cacerts required

This can be achieved using the OPENMETADATA_OPTS environment variable configuration across all the deployments. However, for Production, we recommend you to bundle your cacerts separately for each components (like ElasticSearch/Opensearch and Airflow) and provide that to each individual configs for openmetadata.yaml. You can use this environment variable to also provide extra JVM parameters to tune the application as per your infrastructure needs. Below is an example values to be set for the OPENMETADATA_OPTS environment variable to use cacerts truststore which is bundled for an organization issued certificates -

OPENMETADATA_OPTS="-Djavax.net.ssl.trustStore=<path/to/truststore/file> -Djavax.net.ssl.trustStorePassword=<TRUSTSTORE_PASSWORD>"

It is expected to have the keystore file either mounted as external volume or to be available over the filesystem where openmetadata server application will be running.

Collate 2.0

Breaking Changes by Component

SSL Troubleshooting | Collate Security Guide

SSL Troubleshooting

Bot using JWT as authentication mechanism

Deploying workflows in Airflow

1. Validate the certificate using the public certificate (recommended):

2. Ignore the certification validation (not recommended for production):

Ingesting from CLI

Demo of SSL enabled with an SSO and JWT token configured

Providing a single keystore that has all the cacerts required

​SSL Troubleshooting

​Bot using JWT as authentication mechanism

​Deploying workflows in Airflow

​1. Validate the certificate using the public certificate (recommended):

​2. Ignore the certification validation (not recommended for production):

​Ingesting from CLI

​Demo of SSL enabled with an SSO and JWT token configured

​Providing a single keystore that has all the cacerts required

SSL Troubleshooting

Bot using JWT as authentication mechanism

Deploying workflows in Airflow

1. Validate the certificate using the public certificate (recommended):

2. Ignore the certification validation (not recommended for production):

Ingesting from CLI

Demo of SSL enabled with an SSO and JWT token configured

Providing a single keystore that has all the cacerts required