RELATIONAL-JDBC: Add support for cockroach DB #3352

singhpk234 · 2026-01-05T03:15:42Z

About the change

Add support of cockroach DB in relational-jdbc persistence

There was pr long ago :

Use INT4 type alias for CockroachDB JDBC compatibility #2485

seems like the author got busy, all pending was to testing it E2E with cockroach with all the integ tests along with making it work with schema evolution logic. also marking them as co-author for their contribution.
Happy to close it if they wanna resume their original pr

co-author : @sathesuraj

Checklist

🛡️ Don't disclose security issues! (contact [email protected])
🔗 Clearly explained why the changes are needed, or linked related issues: Fixes #
🧪 Added/updated tests with good coverage, or manually tested (and explained how)
💡 Added comments for complex logic
🧾 Updated CHANGELOG.md (if needed) -- will add later
📚 Updated documentation in site/content/in-dev/unreleased (if needed) - will add later

CockroachDB uses PostgreSQL wire protocol and is indistinguishable from PostgreSQL at the JDBC driver level. Compatibility is ensured through INT4 schema declarations and comprehensive test coverage. All PostgreSQL tests run against both PostgreSQL and CockroachDB.

- Created 5 new integration test classes that mirror PostgreSQL tests - CockroachApplicationIT, CockroachManagementServiceIT, CockroachPolicyServiceIT, CockroachRestCatalogIT, CockroachViewFileIT - All tests extend the same base test classes as PostgreSQL tests - Use CockroachRelationalJdbcProfile for CockroachDB container lifecycle - Tests run with same logic as PostgreSQL to ensure compatibility - Enabled parallel test execution (maxParallelForks = 2) - PostgreSQL and CockroachDB tests can now run concurrently This ensures comprehensive test coverage for CockroachDB backend.

CockroachDB Support: - Added COCKROACHDB to DatabaseType enum (maps to "cockroachdb" directory) - Created separate schema directory: cockroachdb/schema-v1.sql - CockroachDB schema v1 based on PostgreSQL schema v3 (includes all tables) - Uses INT4 explicitly for integer columns (required for CockroachDB JDBC driver) Database Type Detection and Validation: - Implemented DatabaseType.inferFromConnection() to detect database from JDBC metadata - If database-type is configured: uses configured type and validates against connection - If configured type mismatches connection: throws IllegalStateException with clear error - If no configured type: auto-detects from connection (CockroachDB, PostgreSQL, H2) - Validation ensures configuration matches actual database Configuration: - New property: polaris.persistence.relational.jdbc.database-type - Supported values: "postgresql", "cockroachdb", "h2" - If set, configuration is authoritative and validated against connection - If not set, auto-detects from JDBC connection metadata Bootstrap Support: - Added DatabaseType.getLatestSchemaVersion() method - PostgreSQL: latest version is 3 - CockroachDB: latest version is 1 - Updated JdbcBootstrapUtils to use database-specific latest version - Bootstrap automatically selects correct schema for each database type Test Configuration: - CockroachDB test profiles explicitly set database-type=cockroachdb - Ensures proper identification even with PostgreSQL JDBC driver - Added 5 integration test classes for server module - Enabled parallel test execution (PostgreSQL and CockroachDB tests run concurrently) PostgreSQL Schema: - Kept original INTEGER/INT types (no changes to existing schemas) - Removed trailing newlines from schema files

singhpk234 · 2026-01-05T08:14:25Z

...bc/src/main/java/org/apache/polaris/persistence/relational/jdbc/JdbcBasePersistenceImpl.java

+    Set<Long> seenEntityIds = new HashSet<>();
+    for (PolarisBaseEntity entity : entities) {
+      if (!seenEntityIds.add(entity.getId())) {
+        throw new RetryOnConcurrencyException(
+            "Multiple updates to entity id '%s' in the same transaction", entity.getId());
+      }
+    }


Here's the difference and why its required:

PostgreSQL SERIALIZABLE:

Uses Serializable Snapshot Isolation (SSI)

When it detects a write-write conflict within the same transaction, it typically throws SQLSTATE 40001 (serialization_failure) immediately

It's designed to fail fast rather than retry

CockroachDB SERIALIZABLE:

Has automatic retry logic built-in at the database level

When it detects a conflict, it enters an internal retry loop hoping the conflict resolves

For conflicts within the same transaction (unresolvable), it can get stuck retrying

Why / when would this code be invoked with multiple entities having the same ID? 🤔

precisely this test got stuck : https://github.com/apache/polaris/blob/main/integration-tests/src/main/java/org/apache/polaris/service/it/test/PolarisRestCatalogIntegrationBase.java#L1088

i think it more about the implementation of how transaction api is implemented in Polaris

polaris/runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java

Line 1045 in 4cf09ec

commitTransactionRequest.tableChanges().stream()

if you see we could collect all the changes for one table and then apply then one shot rather than making then two calls ?

Interesting ... 🤔

Are you saying that our impl. of commitTransaction() will cause multiple PolarisBaseEntity objects to be written via the Persistence interface, while some of those objects share the same ID?

precisely :) atleast thats my understanding is, we should group by table-uuid collect all the changes and then call commit for individual tables

TBH, that looks like a bug in the commitTransaction() impl. 😅

+1 to grouping changes by table (ID) and writing in one batch.... However, it might be best to make that fix before introducing CockroachDB... just for the sake of clarity.

Here you go : #3360

Sweet - let's merge #3360 , rebase and re-test whether this particular change is still necessary for CockroachDB, if you do not mind.

dimas-b · 2026-01-05T16:20:18Z

co-author : @sathesuraj

Please use the standard Co-Authored-By tag in commits :)

https://docs.github.com/en/pull-requests/committing-changes-to-your-project/creating-and-editing-commits/creating-a-commit-with-multiple-authors

dimas-b

+1 to supporting CockroachDB and thanks for picking this work up, @singhpk234 !

Some comments below.

dimas-b · 2026-01-05T16:25:44Z

...rc/main/java/org/apache/polaris/test/commons/CockroachRelationalJdbcLifeCycleManagement.java

+  @Override
+  public Map<String, String> start() {
+    // CockroachDB testcontainers uses PostgreSQL wire protocol compatibility
+    cockroach = new CockroachContainer("cockroachdb/cockroach:v24.3.0");


Please use ContainerSpecHelper to enable managing docker image versions via Renovate.

Cf.

polaris/runtime/test-common/src/main/java/org/apache/polaris/test/commons/PostgresRelationalJdbcLifeCycleManagement.java

Line 46 in 5d77feb

containerSpecHelper("postgres", PostgresRelationalJdbcLifeCycleManagement.class)

dimas-b · 2026-01-05T16:29:16Z

persistence/relational-jdbc/src/main/resources/cockroachdb/schema-v1.sql

+    PRIMARY KEY (event_id)
+);
+
+-- INT4 type used directly in table definitions for CockroachDB JDBC compatibility


It is probably preferable to move this comment to the top of this file for visibility. WDYT?

dimas-b · 2026-01-05T16:31:32Z

persistence/relational-jdbc/src/main/resources/cockroachdb/schema-v1.sql

Since CockroachDB uses mostly the same Polaris code a PostgreSQL, I believe the schema versions are best kept in sync. I do not think we have to support old schemas in CockroachDB, but the current one should probably be v3 as our PostgreSQL schema... WDYT?

I thought about the same though it make it a bit tricky in sense where we take schema as part of bootstrap, hence thought its better to keep it v1 thought the contents are of v3 sql (as it would just have v3 no v1 / v2).

I additionally also thought how about we do this

ALTER COLUMN FROM INT to INT4 to make it work with existing pg version scripts but it seems like we the feature to enable this

Caused by: java.sql.SQLException: Failed due to 'ERROR: ALTER COLUMN TYPE from int to int4 is only supported experimentally Hint: See: https://go.crdb.dev/issue-v/49329/v24.3

apparently

SET experimental_enable_alter_column_type_general = true; ALTER TABLE your_table_name ALTER COLUMN your_column_name TYPE INT4;

setting this flag would be confusing.
Considering we are above i resorted to v1 name, please let me know your thoughts considering above.

[...] a bit tricky in sense where we take schema as part of bootstrap,

Sorry, @singhpk234 , I do not quite understand this part 🤔 what exactly is "tricky" in case CockroachDB uses v3 for the schema version?

No worries at all (apologies for not being clear), what i meant is what happens some one does bootstrap with v2 in this case, since we give schema version as arg for bootstrapping

polaris/persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/JdbcBootstrapUtils.java

Line 31 in 4cf09ec

* Determines the correct schema version to use for bootstrapping a realm.

may be we can abstract from the user in the handling ?

secondly can be confusing why there is no v1 / v2 unlike other persistence impl.

I believe when we add mariadb / mysql support we would have to work with an assumption that the column name are same and types (for example JSONB vs JSON) are handled accordingly so i just thought of starting fresh with v1 for cockroach which is exactly same as v3 of postgres schema.

but what about this?

polaris/persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/models/ModelEntity.java

Lines 75 to 81 in b1142b5

public static List<String> getAllColumnNames(int schemaVersion) {

if (schemaVersion < 2) {

return ALL_COLUMNS;

} else {

return ALL_COLUMNS_V2;

}

}

Nice catch, i missed it, let me think this more

dimas-b · 2026-01-05T16:38:35Z

...bc/src/main/java/org/apache/polaris/persistence/relational/jdbc/JdbcBasePersistenceImpl.java

+    Set<Long> seenEntityIds = new HashSet<>();
+    for (PolarisBaseEntity entity : entities) {
+      if (!seenEntityIds.add(entity.getId())) {
+        throw new RetryOnConcurrencyException(
+            "Multiple updates to entity id '%s' in the same transaction", entity.getId());
+      }
+    }


Why / when would this code be invoked with multiple entities having the same ID? 🤔

dimas-b · 2026-01-05T16:40:09Z

...bc/src/main/java/org/apache/polaris/persistence/relational/jdbc/JdbcBasePersistenceImpl.java

+    Set<Long> seenEntityIds = new HashSet<>();
+    for (PolarisBaseEntity entity : entities) {
+      if (!seenEntityIds.add(entity.getId())) {
+        throw new RetryOnConcurrencyException(


Why a "retry" exception as opposed to an IllegalArgumentException? I believe the behaviour of this method with multiple objects having the same ID is not well-defined and such situations should be avoided on the caller side.

dimas-b · 2026-01-05T16:50:51Z

...lational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/DatabaseType.java

+  public int getLatestSchemaVersion() {
+    return switch (this) {
+      case POSTGRES -> 3; // PostgreSQL has schemas v1, v2, v3
+      case COCKROACHDB -> 1; // CockroachDB currently has only schema v1
+      case H2 -> 3; // H2 uses same schemas as PostgreSQL


Related to my comment on the schema file: I believe all "latest" JDBC-based schemas should have the same version and evolve in sync. This is because from the caller side (Polaris JDBC persistence code), the "interface" of all JDBC schemas should be the same (column names and type compatibility).

Maintaining different column names and non-trivial type conversions separately for all JDBC backends is going to be too much overhead and will complicate Polaris code unnecessarily.

If the schemas can be assumed to be compatible from the client's (Polaris) perspective, using the same version number will only be natural. WDYT?

Need to think this through more, i went ahead with the current one with the following in the mind :
#3352 (comment)

flyrain · 2026-01-06T00:28:38Z

...in/src/test/java/org/apache/polaris/admintool/relational/jdbc/CockroachJdbcAdminProfile.java

+        "quarkus.datasource.active",
+        "true",
+        "quarkus.datasource.db-kind",
+        "postgresql");


should this be cockroachdb? or quarkus.datasource.db-kind simplifies all postgresql-like ones as postgresql.

Suggested change

"postgresql");

"cockroachdb");

github-project-automation bot moved this to PRs In Progress in Basic Kanban Board Jan 5, 2026

github-project-automation bot added this to Basic Kanban Board Jan 5, 2026

singhpk234 requested review from dimas-b and flyrain January 5, 2026 03:16

sfc-gh-prsingh and others added 6 commits January 4, 2026 20:57

Relational JDBC: Add Cocroach DB compatibility

b16a0bb

spoltessApply

3bec958

fix tests

b25ebae

sfc-gh-prsingh force-pushed the feature/cockroach branch from 5a57873 to b25ebae Compare January 5, 2026 05:04

Handle subtle cocroach behaviour

d19ee00

singhpk234 commented Jan 5, 2026

View reviewed changes

clear up container

0998281

jbonofre self-requested a review January 5, 2026 14:06

dimas-b reviewed Jan 5, 2026

View reviewed changes

flyrain reviewed Jan 6, 2026

View reviewed changes

This was referenced Jan 6, 2026

Fix: Group transaction changes by table #3360

Open

Introduce idempotency records schema and Postgres-backed IdempotencyStore #3205

Merged

	public static List<String> getAllColumnNames(int schemaVersion) {
	if (schemaVersion < 2) {
	return ALL_COLUMNS;
	} else {
	return ALL_COLUMNS_V2;
	}
	}

RELATIONAL-JDBC: Add support for cockroach DB #3352

Are you sure you want to change the base?

RELATIONAL-JDBC: Add support for cockroach DB #3352

Uh oh!

Conversation

singhpk234 commented Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

About the change

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b commented Jan 5, 2026

Uh oh!

dimas-b left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

flyrain Jan 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

singhpk234 commented Jan 5, 2026 •

edited

Loading

dimas-b Jan 5, 2026 •

edited

Loading

dimas-b Jan 5, 2026 •

edited

Loading

flyrain Jan 6, 2026 •

edited

Loading