Creating a Catalog on Apache Polaris

Creating a catalog on Google Cloud Storage (GCS)

Mon, 01 Jan 0001 00:00:00 +0000

For the polaris catalogs create command there are few gcs only options

1--storage-type gcs
2--service-account (Only for GCS) The service account to use when connecting to GCS

example🔗

 1CLIENT_ID=root \
 2CLIENT_SECRET=s3cr3t \
 3DEFAULT_BASE_LOCATION=gs://my-ml-bucket/predictions/ \
 4SERVICE_ACCOUNT=serviceAccount:my-service-account@my-project.iam.gserviceaccount.com \
 5./polaris \
 6 --client-id ${CLIENT_ID} \
 7 --client-secret ${CLIENT_SECRET} \
 8 catalogs \
 9 create \
10 --storage-type gcs \
11 --service-account ${SERVICE_ACCOUNT} \
12 --default-base-location ${DEFAULT_BASE_LOCATION} \
13 my_gcs_catalog

Creating a catalog on Azure

Mon, 01 Jan 0001 00:00:00 +0000

For the polaris catalogs create command there are few azure only options

1--storage-type azure
2--tenant-id (Required for Azure) A tenant ID to use when connecting to Azure Storage
3--multi-tenant-app-name (Only for Azure) The app name to use when connecting to Azure Storage
4--consent-url (Only for Azure) A consent URL granting permissions for the Azure Storage location

Example🔗

 1CLIENT_ID=root \
 2CLIENT_SECRET=s3cr3t \
 3DEFAULT_BASE_LOCATION=abfss://tenant123@blob.core.windows.net \
 4TENANT_ID=tenant123.onmicrosoft.com \
 5MULTI_TENANT_APP_NAME=myapp \
 6CONSENT_URL=https://myapp.com/consent
 7./polaris \
 8 --client-id ${CLIENT_ID} \
 9 --client-secret ${CLIENT_SECRET} \
10 catalogs \
11 create \
12 --storage-type azure \
13 --tenant-id ${TENANT_ID} \
14 --multi-tenant-app-name ${MULTI_TENANT_APP_NAME} \
15 --consent-url ${CONSENT_URL} \
16 --default-base-location ${DEFAULT_BASE_LOCATION} \
17 my_azure_catalog

Migrating from Existing Iceberg Catalogs

Mon, 01 Jan 0001 00:00:00 +0000

There are two ways to migrate an existing Iceberg catalog to Polaris:

Using the Iceberg Catalog Migrator tool: A command-line tool to migrate Iceberg tables from one Iceberg catalog to another. This tool works with any existing Iceberg catalog including Polaris.
Using the Polaris Synchronizer tool: A tool to migrate entities from one Polaris instance to another. This tool is specific to Polaris.