Skip to content

Require SQLAlchemy 2.0 or later for the SQLAlchemy dialect #845

Description

@laughingman7743

Use case

PyAthena declares SQLAlchemy 1.0.0 or later for the sqlalchemy extra, but the dialect and its tests already depend on SQLAlchemy 2.0.
The declared range no longer matches what is tested or what works.

The declarations:

  • pyproject.toml: sqlalchemy = ["sqlalchemy>=1.0.0"] and the dev dependency group "sqlalchemy[asyncio]>=1.0.0".
  • docs/sqlalchemy.md: "Supported SQLAlchemy is 1.0.0 or higher."
  • The aiosqlalchemy extra already requires sqlalchemy[asyncio]>=2.0.0.

What is actually tested and required:

  • CI runs only the locked SQLAlchemy 2.0.46 (uv.lock).
  • The test tree requires 2.0 at import. For example, tests/pyathena/sqlalchemy/test_array.py references types.Double in a module-level parametrize.
  • The dialect reads the 2.0-only TypeEngine._variant_mapping.
    • On master with SQLAlchemy 1.4.54, cast(column("x"), SomeTypeDecorator()) raises AttributeError: 'String' object has no attribute '_variant_mapping'.
    • Measured in an isolated environment.
  • The last 1.4 release is 1.4.54 (2024-09-05).

Proposed change

  • Raise the floor to sqlalchemy>=2.0.0 in the sqlalchemy extra and the dev dependency group, and update docs/sqlalchemy.md.
  • Remove the SQLAlchemy 1.x compatibility code that the new floor makes unreachable:
    • get_double_type() and its hasattr(types, "DOUBLE") fallback to FLOAT.
    • hasattr(types, "Double") / getattr(types, "Double", ...) guards in pyathena/sqlalchemy/compiler.py.
    • The getattr(type_, "_variant_mapping", {}) guard in _ArrayTypeInspector.variant, if Cast to a TypeDecorator as its implementation type #836 is merged first.
  • State the new floor in the release notes.
    Users pinned to SQLAlchemy 1.x would resolve to the last PyAthena release that allows it.

Effect on existing users: environments with SQLAlchemy 1.x installed cannot install the new PyAthena release together with the sqlalchemy extra.
The dialect is untested on 1.x today, and decorated or variant types already fail there.

Out of scope: validating SQLAlchemy 2.1 (the latest release is 2.1.1, while CI tests 2.0.46).

Validation plan (if implementing)

  • just lint.
  • The no-AWS SQLAlchemy tests in tests/pyathena/sqlalchemy/.
  • just test sqla and just test sqla-async in CI.
  • Confirm uv lock still resolves.
  • Check that pip install "PyAthena[sqlalchemy]" with sqlalchemy<2 pinned is rejected.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions