Skip to content

Commit a4f308d

Browse files
authored
Add extra explanation on how to use "--from-pr" and "--from-job" (#45321)
We have the nice feature of being able to reproduce the exact environment most of the tests are run in our CI, but it needs a bit more explanation on how to use it.
1 parent 59bc37f commit a4f308d

File tree

2 files changed

+52
-9
lines changed

2 files changed

+52
-9
lines changed

dev/breeze/doc/06_managing_docker_images.rst

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -120,13 +120,24 @@ To load the image from specific PR, you can use the following command:
120120

121121
.. code-block:: bash
122122
123-
breeze ci-image load --from-pr 12345 --python 3.9 --platform linux/amd64
123+
breeze ci-image load --from-pr 12345 --python 3.9 --github-token <your_github_token>
124124
125125
To load the image from specific job run (for example 12538475388), you can use the following command, find the run id from github action runs.
126126

127127
.. code-block:: bash
128128
129-
breeze ci-image load --from-job 12538475388 --python 3.9 --platform linux/amd64
129+
breeze ci-image load --from-job 12538475388 --python 3.9 --github-token <your_github_token>
130+
131+
After you load the image, you can reproduce the very exact environment that was used in the CI run by
132+
entering breeze container without mounting your local sources:
133+
134+
.. code-block:: bash
135+
136+
breeze shell --mount-sources skip [OTHER OPTIONS]
137+
138+
And you should be able to run any tests and commands interactively in the very exact environment that
139+
was used in the failing CI run. This is a powerful tool to debug and fix CI issues.
140+
130141

131142
.. image:: ./images/image_artifacts.png
132143
:target: https://raw.githubusercontent.com/apache/airflow/main/dev/breeze/images/output_ci-image_load.svg

dev/breeze/doc/ci/07_running_ci_locally.md

Lines changed: 39 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222
**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*
2323

2424
- [Running the CI Jobs locally](#running-the-ci-jobs-locally)
25+
- [Getting the CI image from failing job](#getting-the-ci-image-from-failing-job)
2526
- [Basic variables](#basic-variables)
2627
- [Host & GIT variables](#host--git-variables)
2728
- [In-container environment initialization](#in-container-environment-initialization)
@@ -62,18 +63,49 @@ exactly what our CI is doing by running the sequence of corresponding
6263
In the output of the CI jobs, you will find both - the flags passed and
6364
environment variables set.
6465

66+
# Getting the CI image from failing job
67+
6568
Every contributor can also pull and run images being result of a specific
6669
CI run in GitHub Actions. This is a powerful tool that allows to
6770
reproduce CI failures locally, enter the images and fix them much
68-
faster. It is enough to download and uncompress the artifact that stores the
69-
image and run ``breeze ci-image load -i <path-to-image.tar> --python python``
70-
to load the image and mark the image as refreshed in the local cache.
71+
faster.
72+
73+
Note that this currently only works for AMD machines, not for ARM machines, but
74+
this will change soon.
75+
76+
To load the image from specific PR, you can use the following command:
77+
78+
```bash
79+
breeze ci-image load --from-pr 12345 --python 3.9 --github-token <your_github_token>
80+
```
81+
82+
To load the image from specific job run (for example 12538475388),
83+
you can use the following command, find the run id from github action runs.
84+
85+
```bash
86+
breeze ci-image load --from-job 12538475388 --python 3.9 --github-token <your_github_token>
87+
```
88+
89+
After you load the image, you can reproduce the very exact environment that was used in the CI run by
90+
entering breeze container without mounting your local sources:
91+
92+
```bash
93+
breeze shell --mount-sources skip [OTHER OPTIONS]
94+
```
95+
96+
And you should be able to run any tests and commands interactively in the very exact environment that
97+
was used in the failing CI run even without checking out sources of the failing PR.
98+
This is a powerful tool to debug and fix CI issues.
99+
100+
If you check-out the branch of the PR that was used, regular ``breeze`` commands will
101+
also reproduce the CI environment without having to rebuild the image - for example when dependencies
102+
changed or when new dependencies were released and used in the CI job - and you will
103+
be able to edit source files locally as usual.
104+
71105

72-
You can read more about it in [Breeze](../README.rst) and
73-
[Testing](../../../../contributing-docs/09_testing.rst)
106+
You can read more about it in [Breeze](../README.rst) and [Testing](../../../../contributing-docs/09_testing.rst)
74107

75-
Depending whether the scripts are run locally via
76-
[Breeze](../README.rst) or whether they are run in
108+
Depending whether the scripts are run locally via [Breeze](../README.rst) or whether they are run in
77109
`Build Images` or `Tests` workflows they can take different values.
78110

79111
You can use those variables when you try to reproduce the build locally

0 commit comments

Comments
 (0)