Getting Started with Apache Polaris and Trino

Tags:

This getting started guide provides a docker-compose file to set up Trino with Apache Polaris. Apache Polaris is configured as an Iceberg REST Catalog, and RustFS provides S3-compatible object storage.

The docker-compose file starts the following services:

  • rustfs — S3-compatible object storage for Iceberg data files
  • polaris — Apache Polaris running with an in-memory metastore
  • polaris-setup — bootstraps a catalog, grants, and a namespace in Polaris
  • trino — Trino configured with an Iceberg connector pointing at Polaris

Build the Polaris image🔗

If a Polaris image is not already present locally, build one with the following command:

./gradlew \
   :polaris-server:assemble \
   :polaris-server:quarkusAppPartsBuild --rerun \
   -Dquarkus.container-image.build=true

Run the docker-compose file🔗

Start all services from the repo’s root directory:

docker compose -f site/content/guides/trino/docker-compose.yml up

Wait until the trino service is healthy before proceeding. You can check by watching for the log line HTTP server started in the Trino container output.

Connect to the Trino CLI🔗

Open a Trino CLI session inside the running Trino container:

docker compose -f site/content/guides/trino/docker-compose.yml exec trino trino

Run SQL queries🔗

The polaris-setup service already created a quickstart_catalog backed by RustFS. Create a schema, table, and run some queries:

CREATE SCHEMA polaris.demo;

USE polaris.demo;

CREATE TABLE events (
    event_id   BIGINT,
    event_type VARCHAR,
    user_id    BIGINT,
    created_at TIMESTAMP(6) WITH TIME ZONE
)
WITH (format = 'PARQUET');

INSERT INTO events VALUES
    (1, 'page_view', 101, TIMESTAMP '2024-10-01 10:00:00 UTC'),
    (2, 'click',     102, TIMESTAMP '2024-10-01 11:30:00 UTC'),
    (3, 'purchase',  101, TIMESTAMP '2024-10-02 09:15:00 UTC');

SELECT event_type, count(*) AS cnt
FROM events
GROUP BY event_type
ORDER BY cnt DESC;

Catalog configuration🔗

The Trino Iceberg catalog is defined in catalog/polaris.properties and mounted into the Trino container at /etc/trino/catalog/. Key properties:

PropertyValueDescription
iceberg.catalog.typerestUse the Iceberg REST catalog protocol
iceberg.rest-catalog.urihttp://polaris:8181/api/catalogPolaris REST catalog endpoint
iceberg.rest-catalog.warehousequickstart_catalogPolaris catalog name
iceberg.rest-catalog.securityOAUTH2OAuth2 client credentials flow
iceberg.rest-catalog.http-headersPolaris-Realm: POLARISRoutes requests to the correct Polaris realm
iceberg.rest-catalog.vended-credentials-enabledtrueUse credentials vended by Polaris to access object storage, instead of static S3 credentials
fs.s3.enabledtrueEnable native S3 file system
s3.endpointhttp://rustfs:9000RustFS S3-compatible endpoint
s3.path-style-accesstrueRequired for RustFS

RustFS endpoints🔗

The catalog configuration uses different endpoints for Polaris and Trino versus external clients. endpointInternal (http://rustfs:9000) is used by Polaris when writing Iceberg metadata files. endpoint (http://localhost:9000) is the externally reachable address for clients running outside Docker. Trino configures its S3 endpoint directly in catalog/polaris.properties using the internal Docker hostname rustfs.

Useful URLs🔗