Skip to content

Require SQLAlchemy 2.0 or later for the SQLAlchemy dialect - #858

Merged
laughingman7743 merged 3 commits into
masterfrom
feat/845-sqlalchemy-2-floor
Sep 27, 2026
Merged

laughingman7743 merged 3 commits into
masterfrom
feat/845-sqlalchemy-2-floor

Conversation

@laughingman7743

@laughingman7743 laughingman7743 commented Sep 26, 2026 •

Copy link
Copy Markdown
Member

WHAT

  • Floor raised to SQLAlchemy 2.0.0:
    • pyproject.toml: the sqlalchemy extra and the dev dependency group now require >=2.0.0.
    • uv.lock: updated. Only the two specifier lines change, and no package versions change.
  • Documentation:
    • README.md and docs/introduction.md list >=2.0.0 for the SQLAlchemy extra.
    • docs/sqlalchemy.md states 2.0.0 or higher and drops the "(SQLAlchemy 2.0+)" / "With SQLAlchemy 2.0," qualifiers that implied 1.x support.
  • SQLAlchemy 1.x compatibility branches removed:
    • compiler.py: hasattr(types, "Double") / getattr(types, "Double", get_double_type()) become types.Double, and the true-division casts use types.DOUBLE().
    • base.py: ischema_names["double"] is types.DOUBLE.
    • array.py: _ArrayTypeInspector.variant reads type_._variant_mapping directly.
  • get_double_type() kept: it is exported in pyathena.sqlalchemy.types.__all__, so it stays public, now always returns types.DOUBLE, and is no longer used internally.
  • Tests:
    • test_reflect_table_include_columns drops its 1.2/1.3 branches and uses Inspector.reflect_table.
    • test_numeric_type_variants no longer accepts the 1.x FLOAT fallback for DOUBLE columns.
    • test_get_double_type asserts types.DOUBLE.

WHY

Closes #845.

The declared sqlalchemy>=1.0.0 did not match what is tested or what works:

  • CI tests only the locked SQLAlchemy 2.0.46.
  • The test tree needs 2.0 at import.
  • The dialect relies on the 2.0-only TypeEngine._variant_mapping.

Release note: installing this release with the sqlalchemy extra requires SQLAlchemy 2.0.0 or later.
With sqlalchemy==1.4.54 pinned, uv rejects pyathena[sqlalchemy] as unsatisfiable, so such environments stay on an earlier PyAthena release.

TEST

Tested commit: c687ea4 (code and tests). 349eaeb changes only one line of docs/sqlalchemy.md; just docs lint was rerun on it.

  • just lint: passed.
  • just docs lint: 0 errors.
  • uv run pytest over test_compiler.py, test_array.py, test_temporal.py, test_types.py, test_map.py, and test_struct.py in tests/pyathena/sqlalchemy/, with -k "not Integration": 384 passed.
  • uv run --env-file .env pytest -n 1 tests/pyathena/sqlalchemy/test_base.py -k "test_reflect_table_include_columns or test_reflect_select or test_get_column_type or test_numeric_type_variants" against Athena: 4 passed.
  • uv run --no-project --isolated --with 'sqlalchemy==1.4.54' --with '.[sqlalchemy]': resolution fails.
  • SQLAlchemy 2.0.0 (the new floor) on Python 3.10, in an isolated environment:
    • The dialect imports.
    • cast(x, Double()) renders DOUBLE.
    • AthenaArray(Integer) result processing returns [1].
    • get_double_type() returns types.DOUBLE.
    • A variant cast resolves.
    • SQLAlchemy 2.0.0 itself does not import on Python 3.13, so 2.0.0 was checked on 3.10.

Not run locally: the full just test pyathena, just test sqla, and just test sqla-async suites (left to CI).
SQLAlchemy 2.1 validation is out of scope (#845).

🤖 Generated with Claude Code

laughingman7743 and others added 3 commits September 27, 2026 00:24
The sqlalchemy extra and the dev dependency group declared 1.0.0, but
CI tests only the locked 2.0 release, the test tree needs 2.0 at import,
and the dialect relies on 2.0-only type APIs. Raise the floor to 2.0.0
and drop the documentation qualifiers that implied 1.x support.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With SQLAlchemy 2.0 required, types.DOUBLE, types.Double, and
TypeEngine._variant_mapping always exist. Use them directly and drop the
version-dependent test branches. Keep get_double_type as a public helper
that returns types.DOUBLE.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

if isinstance(left_type, types.Float) or isinstance(right_type, types.Float):
division_type = get_double_type()()
division_type: TypeEngine[Any] = types.DOUBLE()

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review round one — implementation behavior (Claude Code)

Scope: git diff d6891e9e8f4204f5e8ac4d181e7e41814e52332d..349eaeb53b52bce5e3c26db7a50a2b34c1e16982. Covers pyproject.toml, uv.lock, README.md, docs/introduction.md, docs/sqlalchemy.md, array.py, base.py, compiler.py, types.py, test_base.py, and test_types.py.

Result: FINDINGS (1, fixed).

  • Removed guards:
    • types.Double, types.DOUBLE, and TypeEngine._variant_mapping exist in SQLAlchemy 2.0.0.
    • Checked on the floor itself: in an isolated SQLAlchemy 2.0.0 / Python 3.10 environment, the dialect imports, cast(x, Double()) renders DOUBLE, AthenaArray(Integer) results decode, and a variant cast resolves.
  • True division:
    • types.DOUBLE() is the same class get_double_type() returned on 2.x, so the casts are unchanged (test_compiler.py: 158 passed).
    • The explicit TypeEngine[Any] annotation is needed because mypy no longer sees Any.
  • get_double_type() stays in __all__ and returns types.DOUBLE, so imports by users keep working.
  • Tests:
    • test_reflect_table_include_columns keeps the 2.x branch (Inspector.reflect_table) as-is.
    • test_numeric_type_variants narrows the accepted reflected DOUBLE types to the DOUBLE family; ischema_names["double"] is types.DOUBLE.
    • Validation: Athena, 4 passed (-n 1); no-AWS SQLAlchemy tests, 384 passed.
  • Lock: uv lock changed only the two specifiers.
  • Finding, fixed in 349eaeb: docs/sqlalchemy.md:192 still carried "(SQLAlchemy 2.0+)" in the async section. It was redundant under the new floor and inconsistent with the other removed qualifiers.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI note (Claude Code)

Run 36252040653 at 349eaeb, attempt 1: all checks passed except test-sqla-async / run (3.11).

  • In that job, all 577 tests passed (758 skipped). The job failed in session teardown with INTERNALERROR.
  • The failure: the SQLAlchemy test plugin's pytest_testnodedown ran the tests/sqlalchemy/conftest.py drop_db hook, whose DROP DATABASE test_b4798d487ea4 CASCADE failed with SemanticException [Error 10072]: Database does not exist.
  • The log shows no crashed or replaced xdist worker.
  • The same suites passed on the other nine test-sqla / test-sqla-async jobs.
  • This PR does not touch the follower database hooks.
  • Attempt 2 of the failed job passed. All 22 checks now pass; the 3 skipped entries belong to the run from the Draft period.

If the teardown drop failure recurs, the drop_db hook could tolerate a missing follower database; that would be a separate change.

Comment thread pyproject.toml

[project.optional-dependencies]
sqlalchemy = ["sqlalchemy>=1.0.0"]
sqlalchemy = ["sqlalchemy>=2.0.0"]

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review round two — claims, compatibility, evidence (Claude Code)

Scope: d6891e9e8f4204f5e8ac4d181e7e41814e52332d..349eaeb53b52bce5e3c26db7a50a2b34c1e16982, plus the PR body and commit messages.

Result: CLEAN.

Claims checked:

  • Only the specifiers change in the lock: git diff uv.lock shows the two requires-dist / dev specifier lines only.
  • A 1.4 pin is rejected: uv run --no-project --isolated --with 'sqlalchemy==1.4.54' --with '.[sqlalchemy]' reports the requirements as unsatisfiable.
  • SQLAlchemy 2.0.0 on Python 3.13: 2.0.0 fails to import on 3.13 with AssertionError: ... directly inherits TypingOnly but has additional attributes, which comes from SQLAlchemy itself. That is why 2.0.0 was checked on Python 3.10.
  • No other declarations: git grep finds no remaining >=1.0.0 or 1.x statements in README.md, docs/, or pyproject.toml. No benchmarks/ pyproject declares SQLAlchemy.

Caller compatibility:

  • Users on SQLAlchemy 1.x resolve to an earlier PyAthena, which the PR body calls out for the release notes.
  • get_double_type remains importable.
  • No runtime behavior changes on 2.x: every removed branch was the 2.x path already.

Evidence limits: the full pyathena, sqla, and sqla-async suites are left to CI, and SQLAlchemy 2.1 is out of scope.

@@ -37,18 +37,12 @@


def get_double_type() -> type[Any]:

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent review (relayed)

This is a static review of a detached snapshot at the head SHA; the reviewer ran nothing.

  • Reviewer: Codex CLI 0.157.0, model gpt-6-sol, session 01a0de52-df73-7b70-bb91-8084abed18ca.
  • Invocation: codex exec -s read-only --ephemeral.
  • Scope: d6891e9e8f4204f5e8ac4d181e7e41814e52332d..349eaeb53b52bce5e3c26db7a50a2b34c1e16982.
  • The prompt contained the literal diff and constraints only (no PR number, description, or prior findings).

Covered surfaces:

  • The base-to-head diff, and SQLAlchemy 2.x branch equivalence and API use.
  • Callers and tests.
  • pyathena/, tests/, docs/, README.md, pyproject.toml, and benchmarks/ for remaining 1.x assumptions.
  • The public types.__all__, and uv.lock consistency.

The reviewer noted that the changed tests keep the 2.x reflection path and tighten the DOUBLE assertion.

Result: CLEAN — no actionable regression or remaining 1.x-specific requirement.

The review snapshot and PR worktree were unchanged afterwards (git status clean).

@laughingman7743
laughingman7743 marked this pull request as ready for review September 26, 2026 15:28
@laughingman7743
laughingman7743 merged commit 0ba0875 into master Sep 27, 2026
43 of 44 checks passed
@laughingman7743
laughingman7743 deleted the feat/845-sqlalchemy-2-floor branch September 27, 2026 02:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Require SQLAlchemy 2.0 or later for the SQLAlchemy dialect

1 participant