Skip to content

Commit 66c9920

Browse files
lqiu96lqiu96
authored
ci: optimize unit test CI runtime and rename bulk filter (#14029)
## Problem - **Redundant Generated Mock Tests**: Pure GAPIC-generated clients contain thousands of auto-generated mock unit tests that test template echo behavior against in-process mock servers. These tests consume **40+ minutes** per matrix CI run across 240+ client libraries. - **Coverage is Already Provided Elsewhere**: Core generator templates and GAX runtime behaviors are already thoroughly tested by `java-showcase` (`showcase.yaml`), while business logic in handwritten/split libraries is validated in dedicated `split-units` jobs. ## High-Level Changes - **Fast Global Monorepo Compile Pass**: In `.kokoro/build.sh`, dynamically runs `mvn compile -T 1C` for full monorepo CI runs (`units` matrix jobs), reducing full-repository validation from **40+ minutes to ~2 minutes** while keeping all required GitHub status check names. - **Skip Unit Tests on GAPICs by Default**: Configured `<skipUnitTests>true</skipUnitTests>` in `google-cloud-jar-parent/pom.xml` to skip surefire execution for generated modules, while preserving test compilation and enabling `-Pnative` GraalVM test discovery. - **Preserve Testing for Handwritten & Platform Modules**: Explicitly configured `<skipUnitTests>false</skipUnitTests>` on all 18 handwritten/combo library root POMs (`java-storage`, `java-spanner`, `java-bigquery`, `java-bigtable`, `sdk-platform-java`, etc.) so all handwritten unit tests continue to run in `split-units`. - **CI Workflow & Script Optimizations**: Added `cache: maven` to split CI jobs in `.github/workflows/ci.yaml` and multi-threaded execution (`-T 1C`) in `.kokoro/dependencies.sh`.
1 parent f114422 commit 66c9920

27 files changed

Lines changed: 120 additions & 39 deletions

File tree

‎.github/workflows/ci.yaml‎

Lines changed: 22 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -23,8 +23,8 @@ on:
2323
pull_request:
2424
name: ci
2525
jobs:
26-
# detect whether or note we should run "bulk" (non-handwritten) unit tests
27-
bulk-filter:
26+
# detect whether or not we should run generated (non-handwritten) unit tests
27+
generated-libraries-filter:
2828
runs-on: ubuntu-latest
2929
permissions:
3030
pull-requests: read
@@ -57,68 +57,63 @@ jobs:
5757
ci:
5858
- '.github/workflows/ci.yaml'
5959
- '.kokoro/**'
60-
# these unit tests are "bulk" (non-handwritten) libraries
60+
# compile all modules across supported Java versions as a fast smoke test
6161
units:
6262
runs-on: ubuntu-latest
63-
needs: bulk-filter
63+
needs: generated-libraries-filter
64+
if: ${{ needs.generated-libraries-filter.outputs.runnable == 'true' }}
6465
strategy:
6566
fail-fast: false
6667
matrix:
6768
java: [11, 17, 21, 25, 26]
6869
steps:
6970
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
70-
if: ${{ needs.bulk-filter.outputs.runnable == 'true' }}
7171
with:
7272
persist-credentials: false
7373
- uses: actions/setup-java@cf277c60eb25467037889841efdb72551f06f6c3 # v4.9.1
74-
if: ${{ needs.bulk-filter.outputs.runnable == 'true' }}
7574
with:
7675
distribution: temurin
7776
java-version: ${{matrix.java}}
7877
cache: maven
7978
- run: java -version
80-
if: ${{ needs.bulk-filter.outputs.runnable == 'true' }}
8179
- run: .kokoro/build.sh
82-
if: ${{ needs.bulk-filter.outputs.runnable == 'true' && (needs.bulk-filter.outputs.src == 'true' || needs.bulk-filter.outputs.ci == 'true') }}
80+
if: ${{ needs.generated-libraries-filter.outputs.src == 'true' || needs.generated-libraries-filter.outputs.ci == 'true' }}
8381
env:
8482
JOB_TYPE: test
8583
JOB_NAME: units-${{matrix.java}}
8684
units-8-runtime:
8785
runs-on: ubuntu-latest
88-
needs: bulk-filter
86+
needs: generated-libraries-filter
87+
if: ${{ needs.generated-libraries-filter.outputs.runnable == 'true' }}
8988
name: "units (8)"
9089
steps:
9190
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
92-
if: ${{ needs.bulk-filter.outputs.runnable == 'true' }}
9391
with:
9492
persist-credentials: false
9593
- uses: actions/setup-java@cf277c60eb25467037889841efdb72551f06f6c3 # v4.9.1
96-
if: ${{ needs.bulk-filter.outputs.runnable == 'true' }}
9794
with:
9895
java-version: 8
9996
distribution: temurin
10097
- name: "Set jvm system property environment variable for surefire plugin (unit tests)"
10198
# Maven surefire plugin (unit tests) allows us to specify JVM to run the tests.
10299
# https://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#jvm
103-
if: ${{ needs.bulk-filter.outputs.runnable == 'true' }}
104100
run: echo "SUREFIRE_JVM_OPT=-Djvm=${JAVA_HOME}/bin/java" >> $GITHUB_ENV
105101
shell: bash
106102
- uses: actions/setup-java@cf277c60eb25467037889841efdb72551f06f6c3 # v4.9.1
107-
if: ${{ needs.bulk-filter.outputs.runnable == 'true' }}
108103
with:
109104
java-version: 11
110105
distribution: temurin
111106
cache: maven
112107
- run: .kokoro/build.sh
113-
if: ${{ needs.bulk-filter.outputs.runnable == 'true' && (needs.bulk-filter.outputs.src == 'true' || needs.bulk-filter.outputs.ci == 'true') }}
108+
if: ${{ needs.generated-libraries-filter.outputs.src == 'true' || needs.generated-libraries-filter.outputs.ci == 'true' }}
114109
shell: bash
115110
env:
116111
JOB_TYPE: test
117112
JOB_NAME: units-8-runtime-${{matrix.java}}
118113
# detect which libraries have changed
119114
changes:
120-
needs: bulk-filter
121-
if: ${{ needs.bulk-filter.outputs.runnable == 'true' }}
115+
needs: generated-libraries-filter
116+
if: ${{ needs.generated-libraries-filter.outputs.runnable == 'true' }}
122117
runs-on: ubuntu-latest
123118
permissions:
124119
pull-requests: read
@@ -349,7 +344,7 @@ jobs:
349344
env:
350345
BUILD_SUBDIR: ${{matrix.package}}
351346
required:
352-
needs: [ bulk-filter, changes, split-units, split-clirr, split-dependencies ]
347+
needs: [ generated-libraries-filter, changes, split-units, split-clirr, split-dependencies ]
353348
name: conditional-required-check
354349
if: ${{ always() }} # Always run even if any "needs" jobs fail
355350
runs-on: ubuntu-22.04
@@ -360,8 +355,8 @@ jobs:
360355
- name: Success otherwise
361356
run: echo "Success!"
362357
windows:
363-
needs: bulk-filter
364-
if: ${{ needs.bulk-filter.outputs.runnable == 'true' }}
358+
needs: generated-libraries-filter
359+
if: ${{ needs.generated-libraries-filter.outputs.runnable == 'true' }}
365360
runs-on: windows-latest
366361
steps:
367362
- name: Support longpaths
@@ -380,8 +375,8 @@ jobs:
380375
JOB_TYPE: test
381376
JOB_NAME: windows-units
382377
lint:
383-
needs: bulk-filter
384-
if: ${{ needs.bulk-filter.outputs.runnable == 'true' }}
378+
needs: generated-libraries-filter
379+
if: ${{ needs.generated-libraries-filter.outputs.runnable == 'true' }}
385380
runs-on: ubuntu-latest
386381
steps:
387382
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
@@ -400,8 +395,8 @@ jobs:
400395
HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
401396
BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }}
402397
enforcer:
403-
needs: bulk-filter
404-
if: ${{ needs.bulk-filter.outputs.runnable == 'true' }}
398+
needs: generated-libraries-filter
399+
if: ${{ needs.generated-libraries-filter.outputs.runnable == 'true' }}
405400
runs-on: ubuntu-latest
406401
steps:
407402
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
@@ -415,9 +410,9 @@ jobs:
415410
- run: java -version
416411
- run: mvn -B -ntp enforcer:enforce@enforce -T 1C
417412
ban-duplicate-classes:
418-
needs: bulk-filter
413+
needs: generated-libraries-filter
419414
if: |
420-
needs.bulk-filter.outputs.runnable == 'true' &&
415+
needs.generated-libraries-filter.outputs.runnable == 'true' &&
421416
github.head_ref == 'release-please--branches--main' &&
422417
!endsWith(github.event.pull_request.title, 'SNAPSHOT')
423418
runs-on: ubuntu-latest
@@ -438,8 +433,8 @@ jobs:
438433
- run: java -version
439434
- run: mvn -B -ntp enforcer:enforce@enforce-banned-duplicate-classes -T 1C
440435
gapic-libraries-bom:
441-
needs: bulk-filter
442-
if: ${{ needs.bulk-filter.outputs.runnable == 'true' }}
436+
needs: generated-libraries-filter
437+
if: ${{ needs.generated-libraries-filter.outputs.runnable == 'true' }}
443438
runs-on: ubuntu-latest
444439
steps:
445440
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0

‎.kokoro/build.sh‎

Lines changed: 18 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -37,27 +37,39 @@ RETURN_CODE=0
3737

3838
case ${JOB_TYPE} in
3939
test)
40+
MAVEN_GOAL="test"
4041
if [[ -n "${BUILD_SUBDIR}" ]]
4142
then
43+
# Targeted module build (invoked by split-units for handwritten/combo libraries).
44+
# Compiles and runs all unit tests for the specific changed library submodule.
4245
echo "Compiling and building all modules for ${BUILD_SUBDIR}"
4346
install_modules "${BUILD_SUBDIR}"
4447
echo "Running in subdir: ${BUILD_SUBDIR}"
4548
pushd "${BUILD_SUBDIR}"
46-
EXTRA_PROFILE_OPTS=()
49+
EXCLUDE_PROJECTS_OPTS=()
4750
else
48-
EXTRA_PROFILE_OPTS=("-PbulkTests")
51+
# Full monorepo smoke pass across Java runtime matrix versions (units job).
52+
# Pure GAPIC-generated modules skip unit tests by default via <skipUnitTests>true</skipUnitTests>
53+
# in google-cloud-jar-parent; generator correctness and GAX runtime behaviors are covered by
54+
# java-showcase (showcase.yaml). Running a parallel compile pass confirms compatibility
55+
# across all supported JDKs in ~2 minutes instead of 40+ minutes.
56+
MAVEN_GOAL="compile"
57+
# gapic-generator-java is a code generation tool tested in its own dedicated workflow
58+
# (sdk-platform-java-ci.yaml). Excluding it from bulk unit test runs saves 2-3 minutes per
59+
# Java runtime matrix job and avoids reactor dependency resolution race conditions.
60+
EXCLUDE_PROJECTS_OPTS=("--projects" "!sdk-platform-java/gapic-generator-java,!sdk-platform-java/gapic-generator-java-pom-parent")
4961
install_modules "sdk-platform-java"
5062
fi
51-
echo "SUREFIRE_JVM_OPT: ${SUREFIRE_JVM_OPT}"
63+
echo "MAVEN_GOAL: ${MAVEN_GOAL}"
5264
retry_with_backoff 3 10 \
53-
mvn install \
65+
mvn ${MAVEN_GOAL} \
5466
-B -ntp \
5567
-Pquick-build \
5668
-Dorg.slf4j.simpleLogger.showDateTime=true \
5769
-Dorg.slf4j.simpleLogger.dateTimeFormat=HH:mm:ss:SSS \
5870
-Dmaven.wagon.http.retryHandler.count=5 \
59-
--also-make \
60-
${SUREFIRE_JVM_OPT} "${EXTRA_PROFILE_OPTS[@]}"
71+
"${EXCLUDE_PROJECTS_OPTS[@]}" \
72+
-T 1C
6173
RETURN_CODE=$?
6274

6375
if [[ -n "${BUILD_SUBDIR}" ]]

‎.kokoro/common.sh‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -416,6 +416,8 @@ function install_modules() {
416416
printf "Installing submodules:\n%s\n" "$all_submodules"
417417

418418
always_install_deps_list=(
419+
# Required upstream dependency for java-spanner and java-spanner-jdbc
420+
'grpc-gcp-java'
419421
'java-monitoring/google-cloud-monitoring'
420422
'java-monitoring/google-cloud-monitoring-bom'
421423
'java-kms/google-cloud-kms'
@@ -439,6 +441,8 @@ function install_modules() {
439441
'java-iam/proto-google-iam-v3'
440442
'java-iam/proto-google-iam-v3beta'
441443
'gapic-libraries-bom'
444+
# Required upstream dependency for gax-java, google-cloud-core, and all client libraries
445+
'sdk-platform-java/api-common-java'
442446
'sdk-platform-java/java-shared-dependencies'
443447
'sdk-platform-java/java-shared-dependencies/first-party-dependencies'
444448
'sdk-platform-java/java-shared-dependencies/third-party-dependencies'

‎.kokoro/dependencies.sh‎

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -56,11 +56,20 @@ then
5656
pushd "${BUILD_SUBDIR}"
5757
fi
5858

59-
# this should run maven enforcer
60-
mvn install -B -V -ntp \
61-
-Pquick-build -DskipTests=true -Dmaven.javadoc.skip=true -Denforcer.skip=false
59+
# We use the 'test-compile' lifecycle phase (with -DskipTests=true) for the following reasons:
60+
# 1. Why test-compile over compile? 'compile' only builds src/main/java. 'dependency:analyze'
61+
# inspects bytecode in both target/classes and target/test-classes. If test classes are not
62+
# compiled, test-scoped dependencies (such as test mocks and stubs) will be falsely flagged
63+
# as "Unused declared dependencies" by maven-dependency-plugin.
64+
# 2. Why test-compile over package/install? 'package' and 'install' build and bundle JAR archives
65+
# and install them into ~/.m2/repository. Packaging JARs adds significant build time (5-15 mins)
66+
# and is unnecessary for static dependency analysis and enforcer verification. 'test-compile'
67+
# produces the required bytecode in target/classes and target/test-classes in seconds.
68+
# 3. Why -DskipTests=true? Ensures test classes are compiled without executing tests during analysis.
69+
mvn test-compile -B -V -ntp \
70+
-Pquick-build -DskipTests=true -Dmaven.javadoc.skip=true -Denforcer.skip=false -T 1C
6271

63-
mvn -B dependency:analyze -Pquick-build -DfailOnWarning=true -Dmdep.analyze.skip=false
72+
mvn -B dependency:analyze -Pquick-build -DfailOnWarning=true -Dmdep.analyze.skip=false -T 1C
6473

6574
if [[ -n "${BUILD_SUBDIR}" ]]
6675
then

‎google-cloud-jar-parent/pom.xml‎

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,20 @@
1919
<relativePath>../google-cloud-pom-parent/pom.xml</relativePath>
2020
</parent>
2121
<properties>
22-
<skipUnitTests>false</skipUnitTests>
22+
<!--
23+
Defaults to true: most modules under this parent are pure GAPIC-generated clients whose
24+
generated tests aren't worth the CI time. Libraries with handwritten tests worth
25+
running (java-spanner, java-bigtable, java-storage, ...) override this back to false in
26+
their own top-level pom.xml.
27+
28+
Wired exclusively into surefire's <skip> (below):
29+
1. Compiles test sources and dependencies so reactor test-jar dependencies are built
30+
and dependency:analyze accurately recognizes test-scoped usage.
31+
2. Does not suppress failsafe integration tests (ITs).
32+
3. The native profile overrides this back to false so GraalVM native-maven-plugin test
33+
discovery generates required test metadata.
34+
-->
35+
<skipUnitTests>true</skipUnitTests>
2336
<ignoreNonCompile>true</ignoreNonCompile><!-- maven-dependency-plugin:analyze to skip test scope dependencies -->
2437
</properties>
2538

@@ -197,6 +210,13 @@
197210
</plugins>
198211
</build>
199212
</profile>
213+
<profile>
214+
<!-- This profile is used to enable GraalVM native image testing -->
215+
<id>native</id>
216+
<properties>
217+
<skipUnitTests>false</skipUnitTests>
218+
</properties>
219+
</profile>
200220
</profiles>
201221

202222
<build>

‎java-bigquery-jdbc/pom.xml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@
2727
<description>JDBC for BigQuery</description>
2828

2929
<properties>
30+
<skipUnitTests>false</skipUnitTests>
3031
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
3132
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
3233
<github.global.server>github</github.global.server>

‎java-bigquery/pom.xml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@
5151
</licenses>
5252

5353
<properties>
54+
<skipUnitTests>false</skipUnitTests>
5455
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
5556
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
5657
<github.global.server>github</github.global.server>

‎java-bigquerystorage/pom.xml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@
5151
</licenses>
5252

5353
<properties>
54+
<skipUnitTests>false</skipUnitTests>
5455
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
5556
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
5657
<project.protobuf-java.version>3.25.4</project.protobuf-java.version>

‎java-bigtable/pom.xml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,7 @@
139139
</licenses>
140140

141141
<properties>
142+
<skipUnitTests>false</skipUnitTests>
142143
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
143144
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
144145
<github.global.server>github</github.global.server>

‎java-common-protos/pom.xml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@
3939
</licenses>
4040

4141
<properties>
42+
<skipUnitTests>false</skipUnitTests>
4243
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
4344
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
4445
<github.global.server>github</github.global.server>

0 commit comments

Comments
 (0)