Configuring Polaris for Production on Apache Polaris

Configuring GCS Cloud Storage

Mon, 01 Jan 0001 00:00:00 +0000

This page provides guidance for configuring GCS Cloud Storage provider for use with Polaris. It covers credential vending, IAM roles, ACL requirements, and best practices to ensure secure and reliable integration.

All catalog operations in Polaris for Google Cloud Storage (GCS)—including listing, reading, and writing objects—are performed using credential vending, which issues scoped (vended) tokens for secure access.

Polaris requires both IAM roles and Hierarchical Namespace (HNS) ACLs (if HNS is enabled) to be properly configured. Even with the correct IAM role (e.g., roles/storage.objectAdmin), access to paths such as gs://<bucket>/idsp_ns/sample_table4/ may fail with 403 errors if HNS ACLs are missing for scoped tokens. The original access token may work, but scoped (vended) tokens require HNS ACLs on the base path or relevant subpath.

Configuring S3 Storage

Mon, 01 Jan 0001 00:00:00 +0000

This page covers configuring AWS S3, and S3-compatible object stores (MinIO, Apache Ozone S3 gateway, Ceph RGW, and similar), as the storage backend for a Polaris catalog. On AWS S3, all read and write operations are performed using credential vending: Polaris assumes a customer IAM role via STS and returns scoped, short-lived credentials to the client. The IAM role, its trust policy, and the bucket itself must be set up before the catalog is created.

Configuring Azure Blob Cloud Storage

Mon, 01 Jan 0001 00:00:00 +0000

This page covers configuring Azure Blob Storage and Azure Data Lake Storage Gen2 (ADLS Gen2) as the storage backend for a Polaris catalog. Polaris authenticates against Azure with the credentials of a service principal that has data-plane access to the target storage account, and then vends short-lived SAS tokens to clients on each table-load request.

Service principal and Polaris credentials🔗

Polaris uses the Azure SDK’s DefaultAzureCredential chain, which by default reads the service-principal credentials from environment variables. Create a service principal with data access to the storage account and pass its credentials to the Polaris process: