Getting Started with Apache Polaris, Relational JDBC, CockroachDB and Spark SQL

This example requires jq to be installed on your machine.

1. If such an image is not already present, build the Polaris image with support for JDBC persistence and the Postgres JDBC driver:

   1./gradlew \
   2   :polaris-server:assemble \
   3   :polaris-server:quarkusAppPartsBuild --rerun \
   4   :polaris-admin:assemble \
   5   :polaris-admin:quarkusAppPartsBuild --rerun \
   6   -Dquarkus.container-image.build=true
   

2. Start the docker compose group by running the following command from the root of the repository:

   1export POLARIS_PERSISTENCE_RELATIONAL_JDBC_DATABASE_TYPE=cockroachdb
   2export QUARKUS_DATASOURCE_JDBC_URL=jdbc:postgresql://cockroachdb:26257/polaris?sslmode=disable
   3export QUARKUS_DATASOURCE_USERNAME=root
   4export QUARKUS_DATASOURCE_PASSWORD=""
   5export ASSETS_PATH=$(pwd)/site/content/guides/assets/
   6export CLIENT_ID=root
   7export CLIENT_SECRET=s3cr3t
   8docker compose -f site/content/guides/jdbc/docker-compose-bootstrap-db.yml -f site/content/guides/assets/cockroachdb/docker-compose-cockroachdb.yml -f site/content/guides/jdbc/docker-compose.yml up
   

3. Using spark-sql: attach to the running spark-sql container:

   1docker attach $(docker ps -q --filter name=spark-sql)
   

   You may not see Spark’s prompt immediately, type ENTER to see it. A few commands that you can try:

   1CREATE NAMESPACE polaris.ns1;
   2USE polaris.ns1;
   3CREATE TABLE table1 (id int, name string);
   4INSERT INTO table1 VALUES (1, 'a');
   5SELECT * FROM table1;
   

4. To access Polaris from the host machine, first request an access token:

   1export POLARIS_TOKEN=$(curl -s http://localhost:8181/api/catalog/v1/oauth/tokens \
   2   --user root:s3cr3t \
   3   -d 'grant_type=client_credentials' \
   4   -d 'scope=PRINCIPAL_ROLE:ALL' | jq -r .access_token)
   

5. Then, use the access token in the Authorization header when accessing Polaris:

   1curl -v http://localhost:8181/api/management/v1/principal-roles -H "Authorization: Bearer $POLARIS_TOKEN"
   2curl -v http://localhost:8181/api/management/v1/catalogs/quickstart_catalog -H "Authorization: Bearer $POLARIS_TOKEN"
   

6. Using Trino CLI: To access the Trino CLI, run this command:

1docker exec -it jdbc-trino-1 trino

Note, cockroachdb-trino-1 is the name of the Docker container.

Example Trino queries:

SHOW CATALOGS;
SHOW SCHEMAS FROM iceberg;
SHOW TABLES FROM iceberg.information_schema;
DESCRIBE iceberg.information_schema.tables;

CREATE SCHEMA iceberg.tpch;
CREATE TABLE iceberg.tpch.test_polaris AS SELECT 1 x;
SELECT * FROM iceberg.tpch.test_polaris;