Repository navigation
Conversation
This was referenced Oct 20, 2026
Open
Open
Open
This was referenced Oct 20, 2026
nevzheng
force-pushed
the
claude/13900-tc-09-durability
branch
8 times, most recently
from
October 20, 2026 16:22
43001ca to
9bbd259
Compare
Code Coverage Report
|
nevzheng
force-pushed
the
claude/13900-tc-09-durability
branch
from
October 21, 2026 06:19
9bbd259 to
d9af683
Compare
nevzheng
force-pushed
the
claude/13900-tc-09-durability
branch
from
October 21, 2026 07:09
d9af683 to
ce44c3d
Compare
This was referenced Oct 21, 2026
[#14049] improvement(test): Replace the remaining container CLI calls with in-process clients
#14076
Draft
Draft
nevzheng
force-pushed
the
claude/13900-tc-09-durability
branch
from
October 21, 2026 17:20
ce44c3d to
58fd955
Compare
nevzheng
force-pushed
the
claude/13900-tc-09-durability
branch
2 times, most recently
from
October 21, 2026 21:16
9b4090c to
cf1f2a1
Compare
nevzheng
force-pushed
the
claude/13900-tc-09-durability
branch
9 times, most recently
from
October 22, 2026 03:28
1868789 to
c042d82
Compare
…st libraries each module uses
…s built-in store cleanup
Design doc for apache#13900 and a rumdl config for design-docs/testing.
…e ContainerSuite lifecycle - Log every container start with pull, start and readiness times; add test log4j configs so the lines show. - Reset all ContainerSuite state in close(); key Hive by version; make the PostgreSQL retry wait; pin StarRocks; drop a dead start method. - Poll readiness with no initial delay; run cleanDisk() only on real starts. - idp-basic starts its containers once per run; Ryuk back on, unprivileged. - Drop the suite-wide closes in Hive2, Flink and Spark ITs.
…ve readiness without the CLI
… calls with in-process clients
nevzheng
force-pushed
the
claude/13900-tc-09-durability
branch
from
October 22, 2026 04:29
c042d82 to
c1b14ec
Compare
…est containers (part 1/6) ContainerSpec, ContainerExtension and @FixtureContainer: a test declares a container with a fixture annotation and receives it as a parameter or field. Fixtures for Kafka, LocalStack, RustFS, StarRocks, OceanBase and Doris, with their callers migrated.
…o, Ranger and Hive (part 2/6) - TestImages declares every test image in one place. - BaseContainer subclasses can supply their own container (ready for the official Testcontainers modules). - Moto, Ranger and Hive (version, auth) fixtures. - A container whose start fails is closed instead of leaking.
…add ClickHouse fixtures and scope (part 3/6) - client-java, fileset, Hudi, Iceberg, Paimon, Iceberg REST, GVFS, Flink, Spark and Trino env tests use @HiveFixture. - ClickHouse (1 or 3 nodes) and ZooKeeper fixtures. - scope = RUN | CLASS | METHOD; HadoopUserAuthenticationIT gets a private Kerberos Hive.
…and PostgreSQL fixtures (part 4/6) - Review follow-ups from the first fixture changes. - A fixture annotation on a field picks that field's container. - startup = EAGER | LAZY; a warning for fixtures no test injects. - MySQL and PostgreSQL fixtures; SparkQueryRunner starts Hive through its spec. - Only the run's closer closes the container suite. - Split container start time into pull, start and readiness; give the Hive metastore check a real time budget.
…remaining Hive variants (part 5/6) - ContainerSpec.dependencies(): dependencies start first, deepest first, with a cycle check. - ClickHouseCluster injected for the three-node cluster. - Hive auth = RANGER and storage = S3 | GCS | ABS; the CatalogHive IT family and the Ranger ITs use them. - HiveUserAuthenticationIT on the Kerberos Hive fixture.
…th TrinoCluster (part 6/6) TrinoCluster starts the Trino stack through Testcontainers instead of launch.sh and docker compose, with real readiness checks and Ryuk cleanup. @TrinoFixture starts lazily and reads the Gravitino server port from the test.
…ners per fixture - durability = DISPOSABLE (default) | DURABLE on every fixture; each container applies its own recipe at start. - Hive keeps HDFS on tmpfs and its metastore MySQL skips per-commit flushes, for consistency (no measurable gain on a local Mac); ZooKeeper sets forceSync=no; MySQL and PostgreSQL use apache#13899's settings everywhere. - Ranger keeps its policies in the shared MySQL. - -PdurableTestContainers makes every container durable, for debugging. Builds on apache#13899.
nevzheng
force-pushed
the
claude/13900-tc-09-durability
branch
from
October 22, 2026 04:46
c1b14ec to
531b0a5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
durability = DISPOSABLE (default) | DURABLEon every fixture; each container applies its own recipe when it starts. A durable container is private (scope = CLASS | METHOD), which only@HiveFixturewith HDFS storage supports so far; other fixtures fail when the class is found.forceSync=no; MySQL and PostgreSQL use [#13515] improvement(test): Speed up the core MySQL and PostgreSQL test lanes with disposable database data #13899's settings in every module.rangeradminuser limited to its own database.-PdurableTestContainersmakes every container durable, for debugging.This PR includes the commits of the PRs below it. Review only its own commit: nevzheng/gravitino@claude/13900-tc-08-trino-cluster...claude/13900-tc-09-durability It also carries #13899's commits; they drop out once #13899 merges.
Why are the changes needed?
Part of the MySQL gain also comes from replacing Testcontainers' bundled
my.cnf(16 MB buffer pool,table_open_cache = 4), which applies in durable mode too.No test reads data back after a container stops, so syncing to disk is wasted work. #13899 measured MySQL 2.0x and PostgreSQL 1.4x faster on the core lanes; this extends it to every module. Locally, idp-basic (MySQL, PostgreSQL) runs about 55 % faster with disposable data and catalog-jdbc-mysql about 22 %.
Fix: #14003
Does this PR introduce any user-facing change?
No. Test code only.
How was this patch tested?
Unit tests; the same modules run with durability on and off.
Stack: #14103 → #14083 → #14084 → #14085 → #14005 → #14006 → #14052 → #14076 → #14007 → #14008 → #14009 → #14010 → #14011 → #14012 → #14013 → #14079 → #14077 → #14078 → #14086 → #14087 → #14088 → #14089 → #14090 → #14091 → #14094