GH-50253: [CI] Manually install aws-sdk-cpp with brew in order to avoid lock#50254
Conversation
|
|
|
@kou based on: https://docs.brew.sh/Manpage the
That documentation seems outdated and it seems to fail related to this change of default behavior on brew: Applying |
|
@github-actions crossbow submit preview-docs verify-rc-source-*-macos-amd64 |
|
Revision: 9e3445c Submitted crossbow builds: ursacomputing/crossbow @ actions-1c3ad8af77
|
|
The Doc CI related failures are currently failing on main too, I've opened: |
There was a problem hiding this comment.
Pull request overview
This PR addresses recent macOS CI failures caused by Homebrew Bundle running installations in parallel, leading to Homebrew cellar lock conflicts. It restores serialized installs by explicitly setting brew bundle to run with a single job (--jobs 1) across CI and the corresponding developer documentation.
Changes:
- Update GitHub Actions macOS workflows to run
brew bundle ... --jobs 1. - Update the verify-rc macOS task to run
brew bundle ... --jobs 1. - Update developer docs and the GLib README to recommend
brew bundle ... --jobs 1to match CI behavior.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| docs/source/developers/release_verification.rst | Aligns release verification steps with serialized Homebrew Bundle installs. |
| docs/source/developers/python/building.rst | Updates macOS dependency install instructions to use --jobs 1. |
| docs/source/developers/java/building.rst | Updates macOS dependency install instructions to use --jobs 1. |
| docs/source/developers/cpp/building.rst | Updates macOS dependency install instructions to use --jobs 1. |
| dev/tasks/verify-rc/github.macos.yml | Forces serialized Homebrew Bundle installs in verify-rc macOS runs. |
| c_glib/README.md | Updates GLib macOS setup instructions to use --jobs 1. |
| .github/workflows/ruby.yml | Forces serialized Homebrew Bundle installs in Ruby CI on macOS. |
| .github/workflows/python.yml | Forces serialized Homebrew Bundle installs in Python CI on macOS. |
| .github/workflows/cpp.yml | Forces serialized Homebrew Bundle installs in C++ CI on macOS. |
| .github/workflows/cpp_extra.yml | Forces serialized Homebrew Bundle installs in extra C++ macOS workflows. |
|
@tadeja , the llvm issue seems to have been magically fixed here, see the crossbow report above 🤔 |
kou
left a comment
There was a problem hiding this comment.
+1
It makes sense that this is a workaround but can we try the following (install aws-sdk-cpp manually before we run brew bundle`) before we merge this?
brew install aws-sdk-cpp
brew bundle --file=cpp/Brewfile # no --jobs 1And can we report this to Homebrew? I feel that this is not an expected behavior.
Thanks @kou I tried the workaround and it works, see current main.
There seems to be a report in Homebrew already, I've added some more details, see: |
kou
left a comment
There was a problem hiding this comment.
+1
Could you add a comment when we can remove this workaround (with the upstream issue) before you merge this?
|
Failures are unrelated. There are already issues to track all 3 of them |
|
@kou I've added the comment. I'll merge this now. |
Rationale for this change
A change of behavior on the default number of threads used when installing via brew bundle is causing some CI jobs to fail because aws-sdk-cpp is locking
/usr/local/Cellar/cmake.What changes are included in this PR?
Manually install
aws-sdk-cppbefore brew installcpp/BrewfileAre these changes tested?
Yes via CI and archery jobs that were failing.
Are there any user-facing changes?
No.