You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docker_image.yml builds and pushes apache/gluten:ubuntu22-m2-cache (job build-ubuntu22-m2-cache), on every push to main that touches dev/docker/ubuntu/* and on the weekly Sunday cron. I cannot find anything that consumes the resulting image, so I would like to check whether it is still needed before someone spends more time maintaining it.
What I checked, at cd1d1b993:
grep -rn 'm2-cache\|m2cache' over the whole repository returns three hits, all inside docker_image.yml itself: the job name, the file: path, and the tags: value.
The apache/gluten:* images that workflows actually reference are vcpkg-centos-7-gcc13, centos-8-jdk8, centos-9-jdk17, centos-8-jdk17, centos-9-jdk8, centos-9-jdk17-cuda13.1-cudf, vcpkg-centos-9, vcpkg-centos-8-gcc13, vcpkg-centos-7, vcpkg-almalinux-8-gcc13 and centos-7-vcpkg-build-depends. ubuntu22-m2-cache is not among them.
No workflow uses it as a container:, and nothing in .devcontainer/, dev/ or docs/ mentions it.
git log -S 'ubuntu22-m2-cache' --all returns exactly one commit, [VL][CI] Adding docker image for maven cache #11655, which introduced it. So it is not a case of a consumer having been added and later removed.
What each build costs: a Velox dependency setup (builddeps-veloxbe.sh --run_setup_script=ON build_arrow), then one full dependency:go-offline pass over the reactor per supported Spark version, then a push to Docker Hub.
So the question is whether it is consumed somewhere outside this repository, for example by a self-hosted runner setup, a fork, or as a developer convenience that is simply undocumented. If it is, a short note in docs/developers/ would help, since nothing in the tree currently points at it. If it is not, the job looks like a candidate for removal.
Happy to move this to Discussions if that is the better venue.
For context on how this surfaced: #12903 fixes a typo in the same Dockerfile (-Pspark-4 is not a profile id, so that stanza was silently resolving against the root pom defaults instead of Spark 4.0's), and #12902 edits the file as part of removing Spark 3.3 support. Neither depends on the answer here.
This description was written with the assistance of AI tooling (Claude claude-opus-5).
Description
docker_image.ymlbuilds and pushesapache/gluten:ubuntu22-m2-cache(jobbuild-ubuntu22-m2-cache), on every push to main that touchesdev/docker/ubuntu/*and on the weekly Sunday cron. I cannot find anything that consumes the resulting image, so I would like to check whether it is still needed before someone spends more time maintaining it.What I checked, at
cd1d1b993:grep -rn 'm2-cache\|m2cache'over the whole repository returns three hits, all insidedocker_image.ymlitself: the job name, thefile:path, and thetags:value.apache/gluten:*images that workflows actually reference arevcpkg-centos-7-gcc13,centos-8-jdk8,centos-9-jdk17,centos-8-jdk17,centos-9-jdk8,centos-9-jdk17-cuda13.1-cudf,vcpkg-centos-9,vcpkg-centos-8-gcc13,vcpkg-centos-7,vcpkg-almalinux-8-gcc13andcentos-7-vcpkg-build-depends.ubuntu22-m2-cacheis not among them.container:, and nothing in.devcontainer/,dev/ordocs/mentions it.git log -S 'ubuntu22-m2-cache' --allreturns exactly one commit, [VL][CI] Adding docker image for maven cache #11655, which introduced it. So it is not a case of a consumer having been added and later removed.What each build costs: a Velox dependency setup (
builddeps-veloxbe.sh --run_setup_script=ON build_arrow), then one fulldependency:go-offlinepass over the reactor per supported Spark version, then a push to Docker Hub.So the question is whether it is consumed somewhere outside this repository, for example by a self-hosted runner setup, a fork, or as a developer convenience that is simply undocumented. If it is, a short note in
docs/developers/would help, since nothing in the tree currently points at it. If it is not, the job looks like a candidate for removal.Happy to move this to Discussions if that is the better venue.
For context on how this surfaced: #12903 fixes a typo in the same Dockerfile (
-Pspark-4is not a profile id, so that stanza was silently resolving against the root pom defaults instead of Spark 4.0's), and #12902 edits the file as part of removing Spark 3.3 support. Neither depends on the answer here.This description was written with the assistance of AI tooling (Claude claude-opus-5).