Skip to content

Add python 3.13 support #2658

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
+17 −7
Open

Add python 3.13 support #2658

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
acf4dfe
Add python 3.13 support
sa1 Oct 21, 2025
9fcb77d
Merge branch 'apache:main' into iceberg-mirror-sa1
sa1 Oct 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/pypi-build-artifacts.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ jobs:
3.10
3.11
3.12
3.13

- name: Install poetry
run: make install-poetry
Expand All @@ -68,7 +69,7 @@ jobs:
env:
# Ignore 32 bit architectures
CIBW_ARCHS: "auto64"
CIBW_PROJECT_REQUIRES_PYTHON: ">=3.10"
CIBW_PROJECT_REQUIRES_PYTHON: ">=3.10,<=3.13"
Copy link
Contributor

@kevinjqliu kevinjqliu Oct 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: i dont think we need to add an upperbound limit here

CIBW_TEST_REQUIRES: "pytest==7.4.2 moto==5.0.1"
CIBW_TEST_COMMAND: "pytest {project}/tests/avro/test_decoder.py"
# Ignore tests for pypy since not all dependencies are compiled for it
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python: ['3.10', '3.11', '3.12']
python: ['3.10', '3.11', '3.12', '3.13']

steps:
- uses: actions/checkout@v5
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/svn-build-artifacts.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ jobs:
3.10
3.11
3.12
3.13

- name: Install poetry
run: make install-poetry
Expand All @@ -63,7 +64,7 @@ jobs:
env:
# Ignore 32 bit architectures
CIBW_ARCHS: "auto64"
CIBW_PROJECT_REQUIRES_PYTHON: ">=3.10"
CIBW_PROJECT_REQUIRES_PYTHON: ">=3.10,<=3.13"
CIBW_TEST_REQUIRES: "pytest==7.4.2 moto==5.0.1"
CIBW_TEST_COMMAND: "pytest {project}/tests/avro/test_decoder.py"
# Ignore tests for pypy since not all dependencies are compiled for it
Expand Down
8 changes: 7 additions & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ classifiers = [
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
]
packages = [
{ include = "pyiceberg" },
Expand Down Expand Up @@ -66,7 +67,10 @@ pyarrow = { version = ">=17.0.0", optional = true }
google-auth = { version = ">=2.4.0", optional = true }
pandas = { version = ">=1.0.0,<3.0.0", optional = true }
duckdb = { version = ">=0.5.0,<2.0.0", optional = true }
ray = { version = ">=2.10.0,<=2.44.0", optional = true }
ray = [
{ version = ">=2.10.0,<3.0.0", python = ">=3.10", optional = true },
Copy link
Contributor

@geruh geruh Oct 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We used to have a wider upper bound of 3.0.0 like this, and the tight upper bound was added as part of: #2071 (comment).

I was wondering if you were about to have a single entry since on a clean dep resolve Poetry usually chooses the highest compatible version which should technically work across all python releases.

ray = [
  { version = ">=2.45.0,<=2.49.0", python = ">=3.10", optional = true }
]

Copy link
Author

@sa1 sa1 Oct 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this would help people who are stuck on older ray releases. But I can remove it for sure.

{ version = ">=2.45.0", python = ">=3.13", optional = true }
]
python-snappy = { version = ">=0.6.0,<1.0.0", optional = true }
thrift = { version = ">=0.13.0,<1.0.0", optional = true }
boto3 = { version = ">=1.24.59", optional = true }
Expand Down Expand Up @@ -343,6 +347,8 @@ markers = [
# Turns a warning into an error
filterwarnings = [
"error",
# Python 3.13 sqlite3 module ResourceWarnings for unclosed database connections
"ignore:unclosed database in <sqlite3.Connection object*:ResourceWarning",
Copy link
Contributor

@geruh geruh Oct 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you seeing this with Make test?

Copy link
Contributor

@kevinjqliu kevinjqliu Oct 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thats interesting, we should fix this for the tests

Copy link
Author

@sa1 sa1 Oct 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did try to fix the warnings by closing the database, and some of that is in the PR, but they started breaking other legitimate tests so I ignored it like other projects mostly have (mentioned in the issue linked above)

Copy link
Contributor

@kevinjqliu kevinjqliu Oct 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can fix this as a follow up pr

]

[tool.black]
Expand Down
8 changes: 5 additions & 3 deletions tests/catalog/test_base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


from pathlib import PosixPath
from typing import Union
from typing import Generator, Union

import pyarrow as pa
import pytest
Expand Down Expand Up @@ -52,8 +52,10 @@


@pytest.fixture
def catalog(tmp_path: PosixPath) -> InMemoryCatalog:
return InMemoryCatalog("test.in_memory.catalog", **{WAREHOUSE: tmp_path.absolute().as_posix(), "test.key": "test.value"})
def catalog(tmp_path: PosixPath) -> Generator[InMemoryCatalog, None, None]:
catalog = InMemoryCatalog("test.in_memory.catalog", **{WAREHOUSE: tmp_path.absolute().as_posix(), "test.key": "test.value"})
yield catalog
catalog.close()


TEST_TABLE_IDENTIFIER = ("com", "organization", "department", "my_table")
Expand Down
Loading