Persistence

This page describes how to configure persistence for Polaris when deploying with the Helm chart. For general information on persistence in Polaris, see the Metastores page.

Polaris supports three metastore implementations:

• In-Memory - for testing only
• PostgreSQL (JDBC) - recommended for production
• MongoDB (NoSQL) - experimental (beta)

In-Memory Metastore🔗

The in-memory metastore is the default and requires no configuration. It is useful for quick testing and development:

persistence:
  type: in-memory

PostgreSQL (JDBC) Metastore🔗

The PostgreSQL metastore stores all Polaris metadata in a PostgreSQL database. This is the recommended metastore for production deployments.

Prerequisites🔗

• A PostgreSQL database
• Network connectivity from the Kubernetes cluster to the database
• Database credentials with appropriate permissions

Creating the Secret🔗

Create a Kubernetes secret containing the database connection details:

kubectl create secret generic polaris-persistence \
  --namespace polaris \
  --from-literal=username=<db-username> \
  --from-literal=password=<db-password> \
  --from-literal=jdbcUrl=jdbc:postgresql://<db-host>:5432/<db-name>

The secret must contain three keys:

• username: The database username
• password: The database password
• jdbcUrl: The JDBC connection URL

Configuring the Chart🔗

Configure the chart to use the PostgreSQL metastore:

persistence:
  type: relational-jdbc
  relationalJdbc:
    secret:
      name: "polaris-persistence"
      username: "username"      # Key in secret containing the username
      password: "password"      # Key in secret containing the password
      jdbcUrl: "jdbcUrl"        # Key in secret containing the JDBC URL

Connection Pool Settings🔗

For production deployments, you may want to tune the connection pool settings using the advancedConfig section:

advancedConfig:
  quarkus.datasource.jdbc.min-size: "5"
  quarkus.datasource.jdbc.max-size: "20"
  quarkus.datasource.jdbc.acquisition-timeout: "30S"
  quarkus.datasource.jdbc.idle-removal-interval: "5M"

See the Quarkus Datasource configuration reference for all available options.

MongoDB (NoSQL) Metastore🔗

The MongoDB metastore stores all Polaris metadata in a MongoDB database.

Prerequisites🔗

• A MongoDB database
• Network connectivity from the Kubernetes cluster to the database
• A MongoDB connection string with appropriate credentials

Creating the Secret🔗

Create a Kubernetes secret containing the MongoDB connection string, for example:

kubectl create secret generic polaris-mongodb \
  --namespace polaris \
  --from-literal=connectionString='mongodb://<username>:<password>@<host>:<port>'

For MongoDB Atlas or replica sets, use the appropriate connection string format:

# MongoDB Atlas
kubectl create secret generic polaris-mongodb \
  --namespace polaris \
  --from-literal=connectionString='mongodb+srv://<username>:<password>@<cluster>.mongodb.net/?retryWrites=true&w=majority'

# Replica Set
kubectl create secret generic polaris-mongodb \
  --namespace polaris \
  --from-literal=connectionString='mongodb://<username>:<password>@<host1>:<port1>,<host2>:<port2>,<host3>:<port3>/?replicaSet=<rs-name>&authSource=admin'

Configuring the Chart🔗

Configure the chart to use the NoSQL metastore with MongoDb backend:

persistence:
  type: nosql
  nosql:
    backend: MongoDb
    database: polaris
    secret:
      name: "polaris-mongodb"
      connectionString: "connectionString"  # Key in secret containing the connection string

Configuration Options🔗

Advanced MongoDB Configuration🔗

For advanced MongoDB settings, use the advancedConfig section to pass Quarkus MongoDB configuration properties:

advancedConfig:
  quarkus.mongodb.connect-timeout: "10S"
  quarkus.mongodb.read-timeout: "30S"
  quarkus.mongodb.server-selection-timeout: "30S"
  quarkus.mongodb.write-concern.safe: "true"
  quarkus.mongodb.read-preference: "primaryPreferred"

See the Quarkus MongoDB configuration reference for all available options.

Production Recommendations🔗

When deploying Polaris in production, consider the following recommendations:

1. Enable TLS/SSL: Ensure database connections are encrypted. For PostgreSQL, add ?sslmode=require to the JDBC URL. For MongoDB, use tls=true in the connection string.

2. Use strong credentials: Generate strong, unique passwords for database access and rotate them regularly.

3. Configure connection pooling: Tune connection pool settings based on your expected load and database capacity.

4. Set up monitoring: Monitor database performance and connection metrics to identify issues early.

5. Plan for backups: Implement a backup strategy appropriate for your data retention requirements.

6. Test failover: If using a high-availability database setup, test failover scenarios to ensure your application handles them gracefully.