Renames main workflow to Tests (apache#17650)

This is a long-overdue change for CI workflows. Since we are building images in a separate workflow, the `CI Builds` name of the workflow was - first of all misleading, and secondly - too long. The workflow names displayed in the GitHub UI contains the workflow name as prefix so having as short as possible name is an advantage. The `Tests` names seems to be appropriate because this is in fact what we do in this workflow. The change updates the name of workflow as well as documentation that referred to it and fixes a few inconsistencies found in names of the `Build Image` -> `Build Images` workflow. The sequence diagrams showing the CI workflow have been also regenerated with the new name (thanks to mermaid it was super-easy)
diegosolisfr · Aug 18, 2021 · be9911c · be9911c
1 parent 83a2858
commit be9911c
Show file tree

Hide file tree

Showing 20 changed files with 332 additions and 334 deletions.
diff --git a/.github/workflows/build-images.yml b/.github/workflows/build-images.yml
@@ -307,15 +307,15 @@ jobs:
   cancel-on-ci-build:
     permissions:
       actions: write
-    name: "Cancel 'CI Build' jobs on workflow failed/cancelled"
+    name: "Cancel `Tests` jobs on workflow failed/cancelled"
     runs-on: ${{ github.repository == 'apache/airflow' && 'self-hosted' || 'ubuntu-20.04' }}
     if: failure() || cancelled()
     needs: [build-ci-images, build-prod-images]
     env:
       branch: ${{ github.event.pull_request.head.ref }}
       thisRun: ${{ github.run_id }}
     steps:
-      - name: Find running CI Build jobs for ${{ env.TARGET_COMMIT_SHA }}
+      - name: "Find running `Tests` jobs for ${{ env.TARGET_COMMIT_SHA }}"
         run: |
           if [[ "${{ github.event_name }}" == 'pull_request_target' ]]; then
             event_filter="event=pull_request&"

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -16,7 +16,7 @@
 # under the License.
 #
 ---
-name: CI Build
+name: Tests
 on:  # yamllint disable-line rule:truthy
   schedule:
     - cron: '28 0 * * *'
@@ -55,7 +55,7 @@ env:
   # You can switch between building the image in "Build Images" workflow or building them in CI workflow
   # Separately for each job.
   #
-  # a) Using images build in the separate "Build Image" workflow:
+  # a) Using images build in the separate "Build Images" workflow:
   #
   #  GITHUB_REGISTRY_WAIT_FOR_IMAGE: "true"
   #

diff --git a/BREEZE.rst b/BREEZE.rst
@@ -664,7 +664,7 @@ This install additional pypi dependency - torchio in specified version.
      ./breeze build-image --production-image --additional-dev-apt-deps "libasound2-dev" \
         --additional-runtime-apt-deps "libasound2"
 
-This install additional apt dependencies - ``libasound2-dev`` in build image and ``libasound`` in the
+This installs additional apt dependencies - ``libasound2-dev`` in the build image and ``libasound`` in the
 final image. Those are development dependencies that might be needed to build and use python packages added
 via the ``--additional-python-deps`` flag. The ``dev`` dependencies are not installed in the final
 production image, they are only installed in the build "segment" of the production image that is used
@@ -797,8 +797,8 @@ Constraints are generated separately for each python version and there are separ
   providers. If you want to manage airflow separately and then add providers individually, you can
   use those. Use ``no-providers`` mode for that.
 
-In case someone modifies setup.py, the ``CRON`` scheduled CI build automatically upgrades and
-pushes changed to the constraint files, however you can also perform test run of this locally using
+In case someone modifies setup.py, the scheduled CI Tests automatically upgrades and
+pushes changes to the constraint files, however you can also perform test run of this locally using
 the procedure described in `<CONTRIBUTING.rst#manually-generating-constraint-files>`_ which utilises
 multiple processors on your local machine to generate such constraints faster.
 
@@ -1501,7 +1501,7 @@ This is the current syntax for  `./breeze <./breeze>`_:
         Generates pinned constraint files with all extras from setup.py. Those files are generated in
         files folder - separate files for different python version. Those constraint files when
         pushed to orphan constraints-main, constraints-2-0 branches are used
-        to generate repeatable CI builds as well as run repeatable production image builds and
+        to generate repeatable CI test runs as well as run repeatable production image builds and
         upgrades when you want to include installing or updating some of the released providers
         released at the time particular airflow version was released. You can use those
         constraints to predictably install released Airflow versions. This is mainly used to test

diff --git a/CI.rst b/CI.rst
diff --git a/Dockerfile.ci b/Dockerfile.ci
@@ -223,7 +223,7 @@ ARG AIRFLOW_CONSTRAINTS="constraints"
 ARG AIRFLOW_CONSTRAINTS_REFERENCE=""
 ARG AIRFLOW_CONSTRAINTS_LOCATION=""
 ARG DEFAULT_CONSTRAINTS_BRANCH="constraints-main"
-# By changing the CI build epoch we can force reinstalling Airflow and pip all dependencies
+# By changing the epoch we can force reinstalling Airflow and pip all dependencies
 # It can also be overwritten manually by setting the AIRFLOW_CI_BUILD_EPOCH environment variable.
 ARG AIRFLOW_CI_BUILD_EPOCH="3"
 ARG AIRFLOW_PRE_CACHED_PIP_PACKAGES="true"

diff --git a/IMAGES.rst b/IMAGES.rst
@@ -41,7 +41,7 @@ to run Kubernetes tests. See below for the list of arguments that should be prov
 production image from the local sources.
 
 The image is primarily optimised for size of the final image, but also for speed of rebuilds - the
-'airflow-build-image' segment uses the same technique as the CI builds for pre-installing PIP dependencies.
+'airflow-build-image' segment uses the same technique as the CI jobs for pre-installing PIP dependencies.
 It first pre-installs them from the right GitHub branch and only after that final airflow installation is
 done from either local sources or remote location (PIP or GitHub repository).
 
@@ -52,7 +52,7 @@ You can read more details about building, extending and customizing the PROD ima
 CI image
 --------
 
-The CI image is used by `Breeze <BREEZE.rst>`_ as shell image but it is also used during CI build.
+The CI image is used by `Breeze <BREEZE.rst>`_ as the shell image but it is also used during CI tests.
 The image is single segment image that contains Airflow installation with "all" dependencies installed.
 It is optimised for rebuild speed. It installs PIP dependencies from the current branch first -
 so that any changes in setup.py do not trigger reinstalling of all dependencies.
@@ -178,9 +178,9 @@ Using cache during builds
 =========================
 
 Default mechanism used in Breeze for building CI images uses images pulled from
-GitHub Container Registry. This is done to speed up local builds and CI builds - instead of 15 minutes
-for rebuild of CI images, it takes usually less than 3 minutes when cache is used. For CI builds this is
-usually the best strategy - to use default "pull" cache. This is default strategy when
+GitHub Container Registry. This is done to speed up local builds and building images for CI runs - instead of
+> 12 minutes for rebuild of CI images, it takes usually about 1 minute when cache is used.
+For CI images this is usually the best strategy - to use default "pull" cache. This is default strategy when
 `<BREEZE.rst>`_ builds are performed.
 
 For Production Image - which is far smaller and faster to build, it's better to use local build cache (the
@@ -264,7 +264,7 @@ Latest images (pushed when main merge succeeds):
 
 You can see all the current GitHub images at `<https://github.com/apache/airflow/packages>`_
 
-You can read more about the CI configuration and how CI builds are using GitHub images
+You can read more about the CI configuration and how CI jobs are using GitHub images
 in `<CI.rst>`_.
 
 Note that you need to be committer and have the right to push to GitHub and you need to
@@ -305,7 +305,7 @@ Building Python 3.8 CI image using cache pulled from GitHub Container Registry a
 You can also pull and run images being result of a specific CI run in GitHub Actions. This is a powerful
 tool that allows to reproduce CI failures locally, enter the images and fix them much faster. It is enough
 to pass ``--github-image-id`` and the registry and Breeze will download and execute commands using
-the same image that was used during the CI build.
+the same image that was used during the CI tests.
 
 For example this command will run the same Python 3.8 image as was used in build identified with
 9a621eaa394c0a0a336f8e1b31b35eff4e4ee86e commit SHA  with enabled Kerberos integration.
@@ -450,7 +450,7 @@ The following build arguments (``--build-arg`` in docker build command) can be u
 | ``AIRFLOW_PRE_CACHED_PIP_PACKAGES``      | ``true``                                 | Allows to pre-cache airflow PIP packages |
 |                                          |                                          | from the GitHub of Apache Airflow        |
 |                                          |                                          | This allows to optimize iterations for   |
-|                                          |                                          | Image builds and speeds up CI builds     |
+|                                          |                                          | Image builds and speeds up CI jobs       |
 |                                          |                                          | But in some corporate environments it    |
 |                                          |                                          | might be forbidden to download anything  |
 |                                          |                                          | from public repositories.                |
@@ -592,7 +592,7 @@ Refreshing Base Python images
 
 Python base images are updated from time-to-time, usually as a result of implementing security fixes.
 When you build your image locally using ``docker build`` you use the version of image that you have locally.
-For the CI builds using ``breeze`` we use the image that is stored in our repository in order to use cache
+For image builds using ``breeze`` we use the image that is stored in our repository in order to use cache
 efficiently. However CI push build have ``CHECK_IF_BASE_PYTHON_IMAGE_UPDATED`` variable set to ``true``
 which checks if the image has been released and will pull it and rebuild it if needed
 

diff --git a/PULL_REQUEST_WORKFLOW.rst b/PULL_REQUEST_WORKFLOW.rst
@@ -45,7 +45,7 @@ We approached the problem by:
    to control which parts of the tests are run during the tests. This is implemented by the
    ``scripts/ci/selective_ci_checks.sh`` script in our repository. This script analyses which part of the
    code has changed and based on that it sets the right outputs that control which tests are executed in
-   the CI build, and whether we need to build CI images necessary to run those steps. This allowed to
+   the ``Tests`` workflow, and whether we need to build CI images necessary to run those steps. This allowed to
    heavily decrease the strain especially for the Pull Requests that were not touching code (in which case
    the builds can complete in < 2 minutes) but also by limiting the number of tests executed in PRs that do
    not touch the "core" of Airflow, or only touching some - standalone - parts of Airflow such as
@@ -92,7 +92,7 @@ We approached the problem by:
 Selective CI Checks
 -------------------
 
-In order to optimise our CI builds, we've implemented optimisations to only run selected checks for some
+In order to optimise our CI jobs, we've implemented optimisations to only run selected checks for some
 kind of changes. The logic implemented reflects the internal architecture of Airflow 2.0 packages
 and it helps to keep down both the usage of jobs in GitHub Actions as well as CI feedback time to
 contributors in case of simpler changes.

diff --git a/breeze b/breeze
@@ -1830,7 +1830,7 @@ ${CMDNAME} generate-constraints [FLAGS]
       Generates pinned constraint files with all extras from setup.py. Those files are generated in
       files folder - separate files for different python version. Those constraint files when
       pushed to orphan constraints-main, constraints-2-0 branches are used
-      to generate repeatable CI builds as well as run repeatable production image builds and
+      to generate repeatable CI test runs as well as run repeatable production image builds and
       upgrades when you want to include installing or updating some of the released providers
       released at the time particular airflow version was released. You can use those
       constraints to predictably install released Airflow versions. This is mainly used to test
@@ -3550,7 +3550,7 @@ function breeze::run_breeze_command() {
 #######################################################################################################
 #
 # We have different versions of images depending on the python version used. We keep up with the
-# Latest patch-level changes in Python (this is done automatically during CI builds) so we have
+# Latest patch-level changes in Python (this is done automatically during CI test runs) so we have
 # To only take into account MAJOR and MINOR version of python. This variable keeps the major/minor
 # version of python in X.Y format (3.6, 3.7, 3.8, 3.9).
 #

diff --git a/docs/README.rst b/docs/README.rst
@@ -25,8 +25,6 @@ Development documentation preview
 
 Documentation from the development version is built and automatically published: `s.apache.org/airflow-docs <https://s.apache.org/airflow-docs>`_
 
-Documentation for your PRs is available as downloadable artifact in GitHub Actions after the CI builds your PR.
-
 Building documentation
 ======================
 

diff --git a/docs/apache-airflow/extra-packages-ref.rst b/docs/apache-airflow/extra-packages-ref.rst
@@ -301,7 +301,7 @@ Those are extras that install one ore more extras as a bundle.
 +---------------------+-----------------------------------------------------+----------------------------------------------------------------------+
 | devel_all           | ``pip install 'apache-airflow[devel_all]'``         | Everything needed for development (``devel_hadoop`` +  providers)    |
 +---------------------+-----------------------------------------------------+----------------------------------------------------------------------+
-| devel_ci            | ``pip install 'apache-airflow[devel_ci]'``          | All dependencies required for CI build.                              |
+| devel_ci            | ``pip install 'apache-airflow[devel_ci]'``          | All dependencies required for CI tests.                              |
 +---------------------+-----------------------------------------------------+----------------------------------------------------------------------+
 
 Doc extras

diff --git a/images/ci/pull_request_ci_flow.md5 b/images/ci/pull_request_ci_flow.md5
@@ -1 +1 @@
-1c7aa3a8dd16d79c548c52d0390b6152  images/ci/pull_request_ci_flow.mermaid
+7e528f6a1d4b2c01e3a547779f682ee6  images/ci/pull_request_ci_flow.mermaid
diff --git a/images/ci/pull_request_ci_flow.mermaid b/images/ci/pull_request_ci_flow.mermaid
@@ -17,12 +17,12 @@
 
 sequenceDiagram
     Note over Airflow Repo: pull request
-    Note over CI Build: pull_request<br>[Read Token]
+    Note over Tests: pull_request<br>[Read Token]
     Note over Build Images: pull_request_target<br>[Write Token]
     activate Airflow Repo
-    Airflow Repo -->> CI Build: Trigger 'pull_request'
-    activate CI Build
-    CI Build -->> Build Images: Trigger 'pull_request_target'
+    Airflow Repo -->> Tests: Trigger 'pull_request'
+    activate Tests
+    Tests -->> Build Images: Trigger 'pull_request_target'
     activate Build Images
     Note over Build Images: Build info
     par 3.6, [3.7, 3.8, 3.9]
@@ -32,13 +32,13 @@ sequenceDiagram
         Note over Build Images: Build CI Images<br>[COMMIT_SHA]
     end
     par No CI image
-        Note over CI Build: Build info<br>Which tests?<br>Which Python?
+        Note over Tests: Build info<br>Which tests?<br>Which Python?
     and
-        Note over CI Build: OpenAPI client gen
+        Note over Tests: OpenAPI client gen
     and
-        Note over CI Build: Test UI
+        Note over Tests: Test UI
     and
-        Note over CI Build: Test examples<br>PROD image building
+        Note over Tests: Test examples<br>PROD image building
     end
     par 3.6, [3.7, 3.8, 3.9]
         activate GitHub Registry
@@ -51,38 +51,38 @@ sequenceDiagram
     end
     loop Wait for CI images
         par 3.6, [3.7, 3.8, 3.9]
-            CI Build ->> CI Build: Check CI Images
-            Note over CI Build: Wait for<br>[COMMIT_SHA]
+            Tests ->> Tests: Check CI Images
+            Note over Tests: Wait for<br>[COMMIT_SHA]
         end
     end
     par 3.6, [3.7, 3.8, 3.9]
-        GitHub Registry ->> CI Build: Pull CI Image
-        Note over CI Build: Verify CI Image
+        GitHub Registry ->> Tests: Pull CI Image
+        Note over Tests: Verify CI Image
     end
     deactivate GitHub Registry
     par 3.6, [3.7, 3.8, 3.9]
         opt Needed?
-            Note over CI Build: Run static checks
+            Note over Tests: Run static checks
         end
     and
         opt Needed?
-            Note over CI Build: Run basic <br>static checks
+            Note over Tests: Run basic <br>static checks
         end
     and
         opt Needed?
-            Note over CI Build: Build docs
+            Note over Tests: Build docs
         end
     and
         opt Needed?
-            Note over CI Build: Tests
+            Note over Tests: Tests
         end
     and
         opt Needed?
-            Note over CI Build: Test provider <br>packages build
+            Note over Tests: Test provider <br>packages build
         end
     and
         opt Needed?
-            Note over CI Build: Helm tests
+            Note over Tests: Helm tests
         end
     end
     par 3.6, [3.7, 3.8, 3.9]
@@ -93,24 +93,24 @@ sequenceDiagram
     Note over GitHub Registry: Tagged PROD Images<br>[COMMIT_SHA]
     loop Wait for PROD images
         par 3.6, [3.7, 3.8, 3.9]
-            CI Build ->> CI Build: Check PROD Images
-            Note over CI Build: Wait for<br>[COMMIT_SHA]
+            Tests ->> Tests: Check PROD Images
+            Note over Tests: Wait for<br>[COMMIT_SHA]
         end
     end
     par 3.6, [3.7, 3.8, 3.9]
-        GitHub Registry ->> CI Build: Pull PROD Image
-        Note over CI Build: Verify PROD Image
+        GitHub Registry ->> Tests: Pull PROD Image
+        Note over Tests: Verify PROD Image
     end
     deactivate GitHub Registry
     par 3.6, [3.7, 3.8, 3.9]
         opt Needed?
-            Note over CI Build: Run Kubernetes <br>tests
+            Note over Tests: Run Kubernetes <br>tests
         end
     and
         opt Needed?
-            Note over CI Build: Run Kubernetes <br>upgrade tests
+            Note over Tests: Run Kubernetes <br>upgrade tests
         end
     end
-    CI Build -->> Airflow Repo: Status update
+    Tests -->> Airflow Repo: Status update
     deactivate Airflow Repo
-    deactivate CI Build
+    deactivate Tests
diff --git a/images/ci/pull_request_ci_flow.png b/images/ci/pull_request_ci_flow.png
diff --git a/images/ci/push_ci_flow.md5 b/images/ci/push_ci_flow.md5
@@ -1 +1 @@
-b2c2b45380fb4510c7b3325cdb07c2f2  images/ci/push_ci_flow.mermaid
+2000c7ce6fbb164f480c6643d0e0a492  images/ci/push_ci_flow.mermaid
Original file line number	Diff line number	Diff line change
		@@ -1 +1 @@
		1c7aa3a8dd16d79c548c52d0390b6152 images/ci/pull_request_ci_flow.mermaid
		7e528f6a1d4b2c01e3a547779f682ee6 images/ci/pull_request_ci_flow.mermaid
Original file line number	Diff line number	Diff line change
		@@ -1 +1 @@
		b2c2b45380fb4510c7b3325cdb07c2f2 images/ci/push_ci_flow.mermaid
		2000c7ce6fbb164f480c6643d0e0a492 images/ci/push_ci_flow.mermaid