feat: remove python 3.8 support #1215

chalmerlowe · 2025-07-21T14:09:09Z

Removes Python 3.8 support because 3.8 is End of Life.
Includes minor edits to support this change in configs and in the code

Also adds several minor tweaks that were useful for debugging some of the mods to this PR (and are useful in general for all future debugging efforts), such as:

pip freeze to display what dependencies have been installed
* UPDATE: removed upon request: @calculate_duration decorator to confirm the total time to execute a nox session.

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Linchin · 2025-07-23T17:35:37Z

noxfile.py

@@ -56,11 +58,6 @@
    "tests",
 ]
 UNIT_TEST_EXTRAS_BY_PYTHON: Dict[str, List[str]] = {
-    "3.8": [


Do we want to change this to 3.9 instead?

Broadly, the unit & system test extras are not handled well in this (and likely all our noxfiles). As I will show below, we do fully test this library during unit tests, but I think it could be cleaned up (determine which extras need to be done with which Python versions, why, provide comments to clarify the why, and ensure that the logic across the setup.py, owlbot.py, noxfile.py are all in sync, etc.). I recommend that the clean up be a separate Issue as out of scope for the focus of this PR.

For unit tests, our noxfile currently overrides the effect of the variable
UNIT_TEST_EXTRAS_BY_PYTHON. UNIT_TEST_EXTRAS_BY_PYTHON is used in the function install_unittest_dependencies().

In the unit session there is another snippet that also selects extras to be installed. As can be seen below, for any Python version from 3.11-3.13 we test against alembic. But we see that for any Python version outside of 3.11-3.13 we test against all extras, which by definition in setup.py includes alembic.

if install_extras and session.python in ["3.11", "3.12", "3.13"]: install_target = ".[geography,alembic,tests,bqstorage]" elif install_extras: install_target = ".[all]" else: install_target = "." session.install("-e", install_target, "-c", constraints_path)

The truthfulness of this can be see in the Kokoro CI/CD results which show:

the install command: [all] versus [geography,alembic,tests,bqstorage] based on the Python version

the actual install results from pip freeze (which includes alembic in every case)

the successful run for the alembic tests (in every case)

NOTE: I only show two versions, but I hand-checked every version.

3.9 (shortened for space reasons):

nox > Running session unit-3.9(protobuf_implementation='cpp') ... nox > python -m pip install -e '.[all]' -c /tmpfs/src/github/python-bigquery-sqlalchemy/testing/constraints-3.9.txt ... nox > python -m pip freeze alembic==1.16.4 asyncmock==0.4.2 ... tests/unit/test__types.py ........... [ 9%] tests/unit/test_alembic.py .. [ 10%] tests/unit/test_catalog_functions.py ................................... [ 21%]

3.13 (shortened for space reasons):

nox > Running session unit-3.13(protobuf_implementation='upb') ... nox > python -m pip install -e '.[geography,alembic,tests,bqstorage]' -c /tmpfs/src/github/python-bigquery-sqlalchemy/testing/constraints-3.13.txt nox > python -m pip freeze alembic==1.16.4 asyncmock==0.4.2 ... tests/unit/test_alembic.py .. [ 10%] tests/unit/test_catalog_functions.py ................................... [ 21%]

Linchin · 2025-07-23T17:36:18Z

CONTRIBUTING.rst

@@ -148,7 +148,7 @@ Running System Tests

  .. note::

-      System tests are only configured to run under Python 3.8, 3.12, and 3.13.
+      System tests are only configured to run under Python 3.12, and 3.13.


Need to add 3.9 here

Good catch, thanks! Fixed.

Linchin · 2025-07-23T17:43:11Z

noxfile.py

@@ -398,9 +426,7 @@ def compliance(session):
        "-c",
        constraints_path,
    )
-    if session.python == "3.8":
-        extras = "[tests,alembic]"


I wonder if we still need coverage for alembic?

The compliance session selects extras slightly differently than the unit session.
I added alembic to this list of extras to account for that difference. DONE.

I assert that as part of the cleanup task discussed in another comment, all our noxfiles need to be checked to see when/why we choose to do some tests and not others. I suspect there was a historical reason that may OR may not apply.

Linchin · 2025-07-23T17:44:16Z

owlbot.py

@@ -30,14 +30,13 @@
 # ----------------------------------------------------------------------------
 extras = ["tests"]
 extras_by_python = {
-    "3.8": ["tests", "alembic", "bqstorage"],


It feels like we need to change this into 3.9 too?

I added 3.9 with alembic as a system test.

NOTE: there appear to be issues with how owlbot and noxfile, etc are interacting, much like described above for the unit tests. That should be looked into as part of a separate cleanup task.

leahecole · 2025-07-24T14:39:32Z

noxfile.py

@@ -110,6 +102,27 @@

 CURRENT_DIRECTORY = pathlib.Path(__file__).parent.absolute()

+
+def _calculate_duration(func):


If this is not necessary to the 3.8 removal and is a "would be nice", please split into a second PR - I'm happy to be a reviewer for it!

I removed this code.

leahecole · 2025-07-24T14:41:02Z

setup.py

@@ -62,7 +62,7 @@ def readme():
        # https://github.com/grpc/grpc/pull/15254
        "grpcio >= 1.47.0, < 2.0.0",
        "grpcio >= 1.49.1, < 2.0.0; python_version>='3.11'",
-        "pyarrow >= 3.0.0",
+        "pyarrow >= 5.0.0",


Confirming - tests would fail if there were consequences to this +2 major version jump, right?

Tests will fail.
For context: the current version of pyarrow is 20.0
Why version 5.0?:

Version 5.0 came out in 2021.

Version 3.0 and version 4.0 do NOT run on Python 3.9.

Version 5.0 is the first and oldest version that ran on Python 3.9.

This reverts commit 4e86c04.

feat: remove python 3.8 support

6f11d53

chalmerlowe requested review from a team as code owners July 21, 2025 14:09

chalmerlowe requested review from briandorsey and agrawal-siddharth July 21, 2025 14:09

blunderbuss-gcf bot assigned logachev Jul 21, 2025

product-auto-label bot added size: m Pull request size is medium. api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. samples Issues that are directly related to samples. labels Jul 21, 2025

chalmerlowe assigned chalmerlowe and unassigned logachev Jul 21, 2025

chalmerlowe added 2 commits July 23, 2025 12:05

adds debugging to noxfile and tweaks to correct coverage

4e86c04

updates lowest version for system test and owlbot

21fb2ae

chalmerlowe added the kokoro:run Add this label to force Kokoro to re-run the tests. label Jul 23, 2025

updates DEFAULT_PYTHON_VERSION to 3.10

a8fa953

yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Jul 23, 2025

🦉 Updates from OwlBot post-processor

a00b378

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

chalmerlowe requested a review from Linchin July 23, 2025 16:44

chalmerlowe added the automerge Merge the pull request once unit tests and other checks pass. label Jul 23, 2025

Linchin reviewed Jul 23, 2025

View reviewed changes

updates extras in several sessions & lists additional Python ver.

a9881fa

leahecole reviewed Jul 24, 2025

View reviewed changes

chalmerlowe added 3 commits July 24, 2025 15:01

Revert "adds debugging to noxfile and tweaks to correct coverage"

8af7c47

This reverts commit 4e86c04.

adds back in code that was removed

762a46d

adds pragma to avoid cover fallure.

e50cad3

leahecole approved these changes Jul 24, 2025

View reviewed changes

chalmerlowe merged commit 632d6ef into main Jul 24, 2025
15 checks passed

chalmerlowe deleted the remove-python-3.8 branch July 24, 2025 18:22

release-please bot mentioned this pull request Jul 24, 2025

chore(main): release 1.16.0 #1217

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: remove python 3.8 support #1215

feat: remove python 3.8 support #1215

chalmerlowe commented Jul 21, 2025 •

edited

Loading

Uh oh!

Linchin Jul 23, 2025

Uh oh!

chalmerlowe Jul 24, 2025 •

edited

Loading

Uh oh!

Linchin Jul 23, 2025

Uh oh!

chalmerlowe Jul 24, 2025

Uh oh!

Linchin Jul 23, 2025

Uh oh!

chalmerlowe Jul 24, 2025

Uh oh!

Linchin Jul 23, 2025

Uh oh!

chalmerlowe Jul 24, 2025

Uh oh!

leahecole Jul 24, 2025

Uh oh!

chalmerlowe Jul 24, 2025

Uh oh!

leahecole Jul 24, 2025

Uh oh!

chalmerlowe Jul 24, 2025

Uh oh!

Uh oh!

Uh oh!

		@@ -110,6 +102,27 @@

		CURRENT_DIRECTORY = pathlib.Path(__file__).parent.absolute()


		def _calculate_duration(func):

feat: remove python 3.8 support #1215

feat: remove python 3.8 support #1215

Conversation

chalmerlowe commented Jul 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chalmerlowe Jul 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

3.9 (shortened for space reasons):

3.13 (shortened for space reasons):

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

chalmerlowe commented Jul 21, 2025 •

edited

Loading

chalmerlowe Jul 24, 2025 •

edited

Loading