Skip to content

feat: remove python 3.8 support #1215

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jul 24, 2025
Merged

feat: remove python 3.8 support #1215

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
6f11d53
feat: remove python 3.8 support
google-labs-jules[bot] Jul 21, 2025
4e86c04
adds debugging to noxfile and tweaks to correct coverage
chalmerlowe Jul 23, 2025
21fb2ae
updates lowest version for system test and owlbot
chalmerlowe Jul 23, 2025
a8fa953
updates DEFAULT_PYTHON_VERSION to 3.10
chalmerlowe Jul 23, 2025
a00b378
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Jul 23, 2025
a9881fa
updates extras in several sessions & lists additional Python ver.
chalmerlowe Jul 24, 2025
8af7c47
Revert "adds debugging to noxfile and tweaks to correct coverage"
chalmerlowe Jul 24, 2025
762a46d
adds back in code that was removed
chalmerlowe Jul 24, 2025
e50cad3
adds pragma to avoid cover fallure.
chalmerlowe Jul 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/sync-repo-settings.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ branchProtectionRules:
- 'OwlBot Post Processor'
- 'Kokoro'
- 'Samples - Lint'
- 'Samples - Python 3.8'
- 'Samples - Python 3.9'
- 'Samples - Python 3.10'
- 'Samples - Python 3.11'
Expand Down
12 changes: 5 additions & 7 deletions CONTRIBUTING.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ In order to add a feature:
documentation.

- The feature must work fully on the following CPython versions:
3.8, 3.9, 3.10, 3.11, 3.12, and 3.13 on both UNIX and Windows.
3.9, 3.10, 3.11, 3.12, and 3.13 on both UNIX and Windows.

- The feature must not add unnecessary dependencies (where
"unnecessary" is of course subjective, but new dependencies should
Expand Down Expand Up @@ -148,7 +148,7 @@ Running System Tests

.. note::

System tests are only configured to run under Python 3.8, 3.12, and 3.13.
System tests are only configured to run under Python 3.12, and 3.13.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to add 3.9 here

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks! Fixed.

For expediency, we do not run them in older versions of Python 3.

This alone will not run the tests. You'll need to change some local
Expand Down Expand Up @@ -195,11 +195,11 @@ configure them just like the System Tests.

# Run all tests in a folder
$ cd samples/snippets
$ nox -s py-3.8
$ nox -s py-3.9

# Run a single sample test
$ cd samples/snippets
$ nox -s py-3.8 -- -k <name of test>
$ nox -s py-3.9 -- -k <name of test>

********************************************
Note About ``README`` as it pertains to PyPI
Expand All @@ -221,14 +221,12 @@ Supported Python Versions

We support:

- `Python 3.8`_
- `Python 3.9`_
- `Python 3.10`_
- `Python 3.11`_
- `Python 3.12`_
- `Python 3.13`_

.. _Python 3.8: https://docs.python.org/3.8/
.. _Python 3.9: https://docs.python.org/3.9/
.. _Python 3.10: https://docs.python.org/3.10/
.. _Python 3.11: https://docs.python.org/3.11/
Expand All @@ -241,7 +239,7 @@ Supported versions can be found in our ``noxfile.py`` `config`_.
.. _config: https://github.com/googleapis/python-bigquery-sqlalchemy/blob/main/noxfile.py


We also explicitly decided to support Python 3 beginning with version 3.8.
We also explicitly decided to support Python 3 beginning with version 3.9.
Reasons for this include:

- Encouraging use of newest versions of Python 3
Expand Down
2 changes: 1 addition & 1 deletion README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ dependencies.

Supported Python Versions
^^^^^^^^^^^^^^^^^^^^^^^^^
Python >= 3.8, <3.14
Python >= 3.9, <3.14

Unsupported Python Versions
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Expand Down
66 changes: 50 additions & 16 deletions noxfile.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,12 @@

from __future__ import absolute_import

from functools import wraps
import os
import pathlib
import re
import shutil
import time
from typing import Dict, List
import warnings

Expand All @@ -39,9 +41,9 @@
"setup.py",
]

DEFAULT_PYTHON_VERSION = "3.8"
DEFAULT_PYTHON_VERSION = "3.10"

UNIT_TEST_PYTHON_VERSIONS: List[str] = ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]
UNIT_TEST_PYTHON_VERSIONS: List[str] = ["3.9", "3.10", "3.11", "3.12", "3.13"]
UNIT_TEST_STANDARD_DEPENDENCIES = [
"mock",
"asyncmock",
Expand All @@ -56,11 +58,6 @@
"tests",
]
UNIT_TEST_EXTRAS_BY_PYTHON: Dict[str, List[str]] = {
"3.8": [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to change this to 3.9 instead?

Copy link
Collaborator Author

@chalmerlowe chalmerlowe Jul 24, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Broadly, the unit & system test extras are not handled well in this (and likely all our noxfiles). As I will show below, we do fully test this library during unit tests, but I think it could be cleaned up (determine which extras need to be done with which Python versions, why, provide comments to clarify the why, and ensure that the logic across the setup.py, owlbot.py, noxfile.py are all in sync, etc.). I recommend that the clean up be a separate Issue as out of scope for the focus of this PR.

For unit tests, our noxfile currently overrides the effect of the variable
UNIT_TEST_EXTRAS_BY_PYTHON. UNIT_TEST_EXTRAS_BY_PYTHON is used in the function install_unittest_dependencies().

In the unit session there is another snippet that also selects extras to be installed. As can be seen below, for any Python version from 3.11-3.13 we test against alembic. But we see that for any Python version outside of 3.11-3.13 we test against all extras, which by definition in setup.py includes alembic.

if install_extras and session.python in ["3.11", "3.12", "3.13"]:
    install_target = ".[geography,alembic,tests,bqstorage]"
elif install_extras:
    install_target = ".[all]"
else:
    install_target = "."
session.install("-e", install_target, "-c", constraints_path)

The truthfulness of this can be see in the Kokoro CI/CD results which show:

  • the install command: [all] versus [geography,alembic,tests,bqstorage] based on the Python version
  • the actual install results from pip freeze (which includes alembic in every case)
  • the successful run for the alembic tests (in every case)
  • NOTE: I only show two versions, but I hand-checked every version.

3.9 (shortened for space reasons):

nox > Running session unit-3.9(protobuf_implementation='cpp')
...
nox > python -m pip install -e '.[all]' -c /tmpfs/src/github/python-bigquery-sqlalchemy/testing/constraints-3.9.txt
...
nox > python -m pip freeze
alembic==1.16.4
asyncmock==0.4.2
...
tests/unit/test__types.py ...........                                    [  9%]
tests/unit/test_alembic.py ..                                            [ 10%]
tests/unit/test_catalog_functions.py ................................... [ 21%]

3.13 (shortened for space reasons):

nox > Running session unit-3.13(protobuf_implementation='upb')
...
nox > python -m pip install -e '.[geography,alembic,tests,bqstorage]' -c /tmpfs/src/github/python-bigquery-sqlalchemy/testing/constraints-3.13.txt
nox > python -m pip freeze
alembic==1.16.4
asyncmock==0.4.2
...
tests/unit/test_alembic.py ..                                            [ 10%]
tests/unit/test_catalog_functions.py ................................... [ 21%]

"tests",
"alembic",
"bqstorage",
],
"3.11": [
"tests",
"geography",
Expand All @@ -78,7 +75,7 @@
],
}

SYSTEM_TEST_PYTHON_VERSIONS: List[str] = ["3.8", "3.12", "3.13"]
SYSTEM_TEST_PYTHON_VERSIONS: List[str] = ["3.9", "3.12", "3.13"]
SYSTEM_TEST_STANDARD_DEPENDENCIES: List[str] = [
"mock",
"pytest",
Expand All @@ -91,11 +88,6 @@
"tests",
]
SYSTEM_TEST_EXTRAS_BY_PYTHON: Dict[str, List[str]] = {
"3.8": [
"tests",
"alembic",
"bqstorage",
],
"3.12": [
"tests",
"geography",
Expand All @@ -110,6 +102,27 @@

CURRENT_DIRECTORY = pathlib.Path(__file__).parent.absolute()


def _calculate_duration(func):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is not necessary to the 3.8 removal and is a "would be nice", please split into a second PR - I'm happy to be a reviewer for it!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed this code.

"""This decorator prints the execution time for the decorated function."""

@wraps(func)
def wrapper(*args, **kwargs):
start = time.monotonic()
result = func(*args, **kwargs)
end = time.monotonic()
total_seconds = round(end - start)
hours = total_seconds // 3600 # Integer division to get hours
remaining_seconds = total_seconds % 3600 # Modulo to find remaining seconds
minutes = remaining_seconds // 60
seconds = remaining_seconds % 60
human_time = f"{hours:}:{minutes:0>2}:{seconds:0>2}"
print(f"Session ran in {total_seconds} seconds ({human_time})")
return result

return wrapper


nox.options.sessions = [
"unit",
"system",
Expand All @@ -128,13 +141,15 @@


@nox.session(python=DEFAULT_PYTHON_VERSION)
@_calculate_duration
def lint(session):
"""Run linters.

Returns a failure if the linters find linting errors or sufficiently
serious code quality issues.
"""
session.install(FLAKE8_VERSION, BLACK_VERSION)
session.run("python", "-m", "pip", "freeze")
session.run(
"black",
"--check",
Expand All @@ -144,16 +159,19 @@ def lint(session):


@nox.session(python=DEFAULT_PYTHON_VERSION)
@_calculate_duration
def blacken(session):
"""Run black. Format code to uniform standard."""
session.install(BLACK_VERSION)
session.run("python", "-m", "pip", "freeze")
session.run(
"black",
*LINT_PATHS,
)


@nox.session(python=DEFAULT_PYTHON_VERSION)
@_calculate_duration
def format(session):
"""
Run isort to sort imports. Then run black
Expand All @@ -162,6 +180,7 @@ def format(session):
session.install(BLACK_VERSION, ISORT_VERSION)
# Use the --fss option to sort imports using strict alphabetical order.
# See https://pycqa.github.io/isort/docs/configuration/options.html#force-sort-within-sections
session.run("python", "-m", "pip", "freeze")
session.run(
"isort",
"--fss",
Expand All @@ -174,9 +193,11 @@ def format(session):


@nox.session(python=DEFAULT_PYTHON_VERSION)
@_calculate_duration
def lint_setup_py(session):
"""Verify that setup.py is valid (including RST check)."""
session.install("docutils", "pygments")
session.run("python", "-m", "pip", "freeze")
session.run("python", "setup.py", "check", "--restructuredtext", "--strict")


Expand Down Expand Up @@ -213,6 +234,7 @@ def install_unittest_dependencies(session, *constraints):
"protobuf_implementation",
["python", "upb", "cpp"],
)
@_calculate_duration
def unit(session, protobuf_implementation, install_extras=True):
# Install all test dependencies, then install this package in-place.

Expand All @@ -239,6 +261,7 @@ def unit(session, protobuf_implementation, install_extras=True):
session.install("protobuf<4")

# Run py.test against the unit tests.
session.run("python", "-m", "pip", "freeze")
session.run(
"py.test",
"--quiet",
Expand Down Expand Up @@ -288,6 +311,7 @@ def install_systemtest_dependencies(session, *constraints):


@nox.session(python=SYSTEM_TEST_PYTHON_VERSIONS)
@_calculate_duration
def system(session):
"""Run the system test suite."""
constraints_path = str(
Expand All @@ -310,6 +334,7 @@ def system(session):
session.skip("System tests were not found")

install_systemtest_dependencies(session, "-c", constraints_path)
session.run("python", "-m", "pip", "freeze")

# Run py.test against the system tests.
if system_test_exists:
Expand All @@ -331,6 +356,7 @@ def system(session):


@nox.session(python=SYSTEM_TEST_PYTHON_VERSIONS)
@_calculate_duration
def system_noextras(session):
"""Run the system test suite."""
constraints_path = str(
Expand All @@ -355,6 +381,7 @@ def system_noextras(session):
global SYSTEM_TEST_EXTRAS_BY_PYTHON
SYSTEM_TEST_EXTRAS_BY_PYTHON = False
install_systemtest_dependencies(session, "-c", constraints_path)
session.run("python", "-m", "pip", "freeze")

# Run py.test against the system tests.
if system_test_exists:
Expand All @@ -376,6 +403,7 @@ def system_noextras(session):


@nox.session(python=SYSTEM_TEST_PYTHON_VERSIONS[-1])
@_calculate_duration
def compliance(session):
"""Run the SQLAlchemy dialect-compliance system tests"""
constraints_path = str(
Expand All @@ -398,9 +426,7 @@ def compliance(session):
"-c",
constraints_path,
)
if session.python == "3.8":
extras = "[tests,alembic]"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we still need coverage for alembic?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The compliance session selects extras slightly differently than the unit session.
I added alembic to this list of extras to account for that difference. DONE.

I assert that as part of the cleanup task discussed in another comment, all our noxfiles need to be checked to see when/why we choose to do some tests and not others. I suspect there was a historical reason that may OR may not apply.

elif session.python in ["3.12", "3.13"]:
if session.python in ["3.12", "3.13"]:
extras = "[tests,geography]"
else:
extras = "[tests]"
Expand Down Expand Up @@ -430,19 +456,22 @@ def compliance(session):


@nox.session(python=DEFAULT_PYTHON_VERSION)
@_calculate_duration
def cover(session):
"""Run the final coverage report.

This outputs the coverage report aggregating coverage from the unit
test runs (not system test runs), and then erases coverage data.
"""
session.install("coverage", "pytest-cov")
session.run("python", "-m", "pip", "freeze")
session.run("coverage", "report", "--show-missing", "--fail-under=100")

session.run("coverage", "erase")


@nox.session(python="3.10")
@_calculate_duration
def docs(session):
"""Build the docs for this library."""

Expand All @@ -465,6 +494,7 @@ def docs(session):
)

shutil.rmtree(os.path.join("docs", "_build"), ignore_errors=True)
session.run("python", "-m", "pip", "freeze")
session.run(
"sphinx-build",
"-W", # warnings as errors
Expand All @@ -480,6 +510,7 @@ def docs(session):


@nox.session(python="3.10")
@_calculate_duration
def docfx(session):
"""Build the docfx yaml files for this library."""

Expand All @@ -502,6 +533,7 @@ def docfx(session):
)

shutil.rmtree(os.path.join("docs", "_build"), ignore_errors=True)
session.run("python", "-m", "pip", "freeze")
session.run(
"sphinx-build",
"-T", # show full traceback on exception
Expand Down Expand Up @@ -532,6 +564,7 @@ def docfx(session):
"protobuf_implementation",
["python", "upb", "cpp"],
)
@_calculate_duration
def prerelease_deps(session, protobuf_implementation):
"""Run all tests with prerelease versions of dependencies installed."""

Expand Down Expand Up @@ -593,6 +626,7 @@ def prerelease_deps(session, protobuf_implementation):
"requests",
]
session.install(*other_deps)
session.run("python", "-m", "pip", "freeze")

# Print out prerelease package versions
session.run(
Expand Down
5 changes: 2 additions & 3 deletions owlbot.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,13 @@
# ----------------------------------------------------------------------------
extras = ["tests"]
extras_by_python = {
"3.8": ["tests", "alembic", "bqstorage"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels like we need to change this into 3.9 too?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added 3.9 with alembic as a system test.

NOTE: there appear to be issues with how owlbot and noxfile, etc are interacting, much like described above for the unit tests. That should be looked into as part of a separate cleanup task.

"3.11": ["tests", "geography", "bqstorage"],
"3.12": ["tests", "geography", "bqstorage"],
"3.13": ["tests", "geography", "bqstorage"],
}
templated_files = common.py_library(
unit_test_python_versions=["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"],
system_test_python_versions=["3.8", "3.12", "3.13"],
unit_test_python_versions=["3.9", "3.10", "3.11", "3.12", "3.13"],
system_test_python_versions=["3.9", "3.12", "3.13"],
cov_level=100,
unit_test_extras=extras,
unit_test_extras_by_python=extras_by_python,
Expand Down
3 changes: 0 additions & 3 deletions samples/snippets/requirements-test.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,13 @@ google-auth==2.40.3
google-cloud-testutils==1.6.4
iniconfig==2.1.0
packaging==25.0
pluggy===1.5.0; python_version == '3.8'
pluggy==1.6.0; python_version >= '3.9'
py==1.11.0
pyasn1==0.6.1
pyasn1-modules==0.4.2
pyparsing===3.1.4; python_version == '3.8'
pyparsing==3.2.3; python_version >= '3.9'
pytest===6.2.5
rsa==4.9.1
six==1.17.0
toml==0.10.2
typing-extensions===4.13.0; python_version == '3.8'
typing-extensions==4.14.1; python_version >= '3.9'
Loading