Repository navigation
Conversation
This was referenced Oct 21, 2026
Open
Open
[#14049] improvement(test): Replace the remaining container CLI calls with in-process clients
#14076
Draft
Open
Open
Open
Draft
Draft
Draft
nevzheng
force-pushed
the
claude/13900-tc-20-no-exec
branch
7 times, most recently
from
October 21, 2026 19:28
1685487 to
b08e900
Compare
nevzheng
force-pushed
the
claude/13900-tc-20-no-exec
branch
4 times, most recently
from
October 22, 2026 03:28
6c02410 to
408df46
Compare
…st libraries each module uses
…s built-in store cleanup
Design doc for apache#13900 and a rumdl config for design-docs/testing.
…e ContainerSuite lifecycle - Log every container start with pull, start and readiness times; add test log4j configs so the lines show. - Reset all ContainerSuite state in close(); key Hive by version; make the PostgreSQL retry wait; pin StarRocks; drop a dead start method. - Poll readiness with no initial delay; run cleanDisk() only on real starts. - idp-basic starts its containers once per run; Ryuk back on, unprivileged. - Drop the suite-wide closes in Hive2, Flink and Spark ITs.
…ve readiness without the CLI
… calls with in-process clients
nevzheng
force-pushed
the
claude/13900-tc-20-no-exec
branch
from
October 22, 2026 04:29
408df46 to
9aee780
Compare
…est containers (part 1/6) ContainerSpec, ContainerExtension and @FixtureContainer: a test declares a container with a fixture annotation and receives it as a parameter or field. Fixtures for Kafka, LocalStack, RustFS, StarRocks, OceanBase and Doris, with their callers migrated.
…o, Ranger and Hive (part 2/6) - TestImages declares every test image in one place. - BaseContainer subclasses can supply their own container (ready for the official Testcontainers modules). - Moto, Ranger and Hive (version, auth) fixtures. - A container whose start fails is closed instead of leaking.
…add ClickHouse fixtures and scope (part 3/6) - client-java, fileset, Hudi, Iceberg, Paimon, Iceberg REST, GVFS, Flink, Spark and Trino env tests use @HiveFixture. - ClickHouse (1 or 3 nodes) and ZooKeeper fixtures. - scope = RUN | CLASS | METHOD; HadoopUserAuthenticationIT gets a private Kerberos Hive.
…and PostgreSQL fixtures (part 4/6) - Review follow-ups from the first fixture changes. - A fixture annotation on a field picks that field's container. - startup = EAGER | LAZY; a warning for fixtures no test injects. - MySQL and PostgreSQL fixtures; SparkQueryRunner starts Hive through its spec. - Only the run's closer closes the container suite. - Split container start time into pull, start and readiness; give the Hive metastore check a real time budget.
…remaining Hive variants (part 5/6) - ContainerSpec.dependencies(): dependencies start first, deepest first, with a cycle check. - ClickHouseCluster injected for the three-node cluster. - Hive auth = RANGER and storage = S3 | GCS | ABS; the CatalogHive IT family and the Ranger ITs use them. - HiveUserAuthenticationIT on the Kerberos Hive fixture.
…th TrinoCluster (part 6/6) TrinoCluster starts the Trino stack through Testcontainers instead of launch.sh and docker compose, with real readiness checks and Ryuk cleanup. @TrinoFixture starts lazily and reads the Gravitino server port from the test.
…ners per fixture - durability = DISPOSABLE (default) | DURABLE on every fixture; each container applies its own recipe at start. - Hive keeps HDFS on tmpfs and its metastore MySQL skips per-commit flushes, for consistency (no measurable gain on a local Mac); ZooKeeper sets forceSync=no; MySQL and PostgreSQL use apache#13899's settings everywhere. - Ranger keeps its policies in the shared MySQL. - -PdurableTestContainers makes every container durable, for debugging. Builds on apache#13899.
…-private shared containers Every test now gets its containers through a spec. Test classes declare fixture annotations; code that cannot carry one (BaseIT's Gravitino backend, which the core database lanes use, idp-basic's backend parameter, RangerITEnv, SparkQueryRunner) calls Fixtures.get(spec), and tests that build a container of their own use Fixtures.network(). ContainerSuite becomes the package-private SharedContainers, used only by specs and ContainerExtension: the same images, environment, host names, network, one container per key per run, run-end close and timing lines. Its unused starts (the single Trino container, default Doris and PostgreSQL overloads) go away; disabled TrinoConnectorIT builds its Trino container itself. TestContainerSuite becomes TestSharedContainers.
…ork per-run names
…containers wait strategies Move the post-start poll loops of Hive, Doris, Ranger, StarRocks, OceanBase, RustFS, ClickHouse, Kafka and Trino into wait strategies: the port or log check composed with a ProbeWaitStrategy for checks that need a client. Startup timeouts are the old port budget plus the old poll budget. Readiness time now shows in containerMs of the [CONTAINER-TIMING] line.
…nd OceanBase on official Testcontainers modules
…tainers onto fixtures
…ontainer and durable data
… for it without commands in the container
…ners for readiness and logs
nevzheng
force-pushed
the
claude/13900-tc-20-no-exec
branch
from
October 22, 2026 04:46
9aee780 to
b132895
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
KerberosHdfshelper with the owner and mode the old CLI step produced, and metastore port checks; the JVM's Kerberos settings are left as they were.tarin the container; archive names include the container's per-run name, so a private container no longer overwrites the shared one's logs.DatabaseContainerreads its tmpfs mounts from Docker instead of runningstat.The only command left inside a container is
HadoopUserAuthenticationIT'skadmin.local; the next PR, #14094, moves it into the container's own start.Why are the changes needed?
It completes "no CLIs in tests" for the framework itself.
Fix: #14049
Does this PR introduce any user-facing change?
No. Test code only.
How was this patch tested?
Unit tests for the Kerberos helper and log naming; the Kerberos Hive, fileset, Iceberg, Paimon and Hudi ITs, and Kafka/Doris/Trino log copies.
Stack: #14103 → #14083 → #14084 → #14085 → #14005 → #14006 → #14052 → #14076 → #14007 → #14008 → #14009 → #14010 → #14011 → #14012 → #14013 → #14079 → #14077 → #14078 → #14086 → #14087 → #14088 → #14089 → #14090 → #14091 → #14094