Skip to content

Support PyTest container test suite for Python Container. #768

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Dec 4, 2025
Merged

Support PyTest container test suite for Python Container. #768

merged 8 commits into from
Dec 4, 2025
+805 −276

Conversation

@phracek
Copy link
Member

@phracek phracek commented Nov 11, 2025 •
edited by github-actions bot
Loading

This pull request migrates bash test suite
to container-ci-suite stored here: https://github.org/sclorg/container-ci-suite

Introduce PyTest suite migrated from test/run bash suite.
Migration matrix is:

build_s2i_app is fixture that takes Path test-app from conftest.py
file.

test_container_application.py tests all test-app directories.

test_container_basics.py tests only basics w/o application.
It also tests test_dockerfile and test_minimal_dockerfile

Signed-off-by: Petr "Stone" Hracek [email protected]

phracek added a commit to phracek/s2i-python-container that referenced this pull request Nov 11, 2025
@phracek
Enables testing s2i-python-container by PyTest suite.
a0ea5da 
This blocks: sclorg#768

Signed-off-by: Petr "Stone" Hracek <[email protected]>
@phracek phracek mentioned this pull request Nov 11, 2025
Merged
phracek added a commit that referenced this pull request Nov 11, 2025
@phracek
Enables testing s2i-python-container by PyTest suite.
6d0f7d4 
This blocks: #768

Signed-off-by: Petr "Stone" Hracek <[email protected]>
@phracek
Copy link
Member Author

phracek commented Nov 11, 2025

Rebased against master. Let's run first round of tests.
[test][test-pytest][test-openshift-pytest]

@github-actions
Copy link

github-actions bot commented Nov 11, 2025 •
edited
Loading

Testing Farm results

namecomposearchstatusstarted (UTC)timelogs
RHEL8 - PyTest - OpenShift 4 - 3.12RHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 15:49:2027min 15stest pipeline
RHEL8 - PyTest - OpenShift 4 - 3.9RHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 15:43:3728min 8stest pipeline
RHEL8 - 3.9-minimalRHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 16:18:0425min 19stest pipeline
RHEL8 - PyTest - 3.11RHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 15:24:5122min 25stest pipeline
RHEL8 - PyTest - OpenShift 4 - 3.12-minimalRHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 15:45:2325min 18stest pipeline
RHEL8 - PyTest - 3.9RHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 15:24:5122min 26stest pipeline
RHEL8 - PyTest - 3.12-minimalRHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 15:25:2521min 21stest pipeline
RHEL10 - PyTest - OpenShift 4 - 3.12-minimalRHEL-10-Nightlyx86_64✅ passed02.12.2025 15:53:1323min 26stest pipeline
RHEL8 - PyTest - OpenShift 4 - 3.11RHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 15:24:5831min 14stest pipeline
RHEL10 - PyTest - 3.12-minimalRHEL-10-Nightlyx86_64✅ passed02.12.2025 15:47:4615min 38stest pipeline
RHEL8 - PyTest - 3.9-minimalRHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 15:24:5221min 54stest pipeline
RHEL9 - PyTest - OpenShift 4 - 3.12RHEL-9.6.0-Nightlyx86_64✅ passed02.12.2025 15:57:2234min 51stest pipeline
RHEL8 - PyTest - 3.11-minimalRHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 15:24:4922min 10stest pipeline
RHEL9 - 3.9RHEL-9.6.0-Nightlyx86_64✅ passed18.11.2025 13:51:5130min 25stest pipeline
RHEL9 - 3.12-minimalRHEL-9.6.0-Nightlyx86_64✅ passed02.12.2025 16:11:4528min 10stest pipeline
RHEL9 - PyTest - 3.12-minimalRHEL-9.6.0-Nightlyx86_64✅ passed02.12.2025 15:48:0524min 57stest pipeline
RHEL9 - PyTest - OpenShift 4 - 3.12-minimalRHEL-9.6.0-Nightlyx86_64✅ passed02.12.2025 15:48:0630min 27stest pipeline
RHEL9 - PyTest - OpenShift 4 - 3.11RHEL-9.6.0-Nightlyx86_64✅ passed02.12.2025 15:25:0933min 42stest pipeline
RHEL8 - 3.11RHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 16:16:2927min 53stest pipeline
RHEL9 - PyTest - 3.9RHEL-9.6.0-Nightlyx86_64✅ passed02.12.2025 15:24:5124min 55stest pipeline
RHEL9 - PyTest - OpenShift 4 - 3.9RHEL-9.6.0-Nightlyx86_64✅ passed02.12.2025 15:25:2033min 43stest pipeline
RHEL9 - PyTest - 3.11RHEL-9.6.0-Nightlyx86_64✅ passed02.12.2025 15:24:4826min 48stest pipeline
RHEL8 - 3.12RHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 15:49:2927min 26stest pipeline
RHEL8 - 3.12-minimalRHEL-8.10.0-Nightlyx86_64✅ passed03.12.2025 14:07:2726min 38stest pipeline
RHEL10 - 3.12-minimalRHEL-10-Nightlyx86_64✅ passed02.12.2025 15:51:2921min 29stest pipeline
CentOS Stream 9 - PyTest - 3.9CentOS-Stream-9x86_64✅ passed02.12.2025 15:43:0717min 50stest pipeline
CentOS Stream 10 - PyTest - 3.12-minimalCentOS-Stream-10x86_64✅ passed02.12.2025 15:24:5013min 10stest pipeline
CentOS Stream 9 - PyTest - 3.11-minimalCentOS-Stream-9x86_64✅ passed02.12.2025 15:49:5215min 39stest pipeline
CentOS Stream 9 - PyTest - 3.12-minimalCentOS-Stream-9x86_64✅ passed02.12.2025 15:49:1515min 20stest pipeline
CentOS Stream 10 - 3.12CentOS-Stream-10x86_64✅ passed02.12.2025 16:07:4420min 26stest pipeline
RHEL9 - PyTest - 3.12RHEL-9.6.0-Nightlyx86_64✅ passed02.12.2025 15:39:1426min 42stest pipeline
RHEL8 - PyTest - 3.12RHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 15:25:0921min 35stest pipeline
RHEL8 - 3.11-minimalRHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 16:16:0225min 18stest pipeline
RHEL8 - 3.9RHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 16:05:4327min 9stest pipeline
Fedora - PyTest - 3.13Fedora-latestx86_64✅ passed02.12.2025 16:01:3615min 46stest pipeline
CentOS Stream 9 - PyTest - 3.9-minimalCentOS-Stream-9x86_64✅ passed02.12.2025 15:59:5715min 18stest pipeline
CentOS Stream 10 - PyTest - 3.12CentOS-Stream-10x86_64✅ passed02.12.2025 15:47:2618min 16stest pipeline
CentOS Stream 9 - 3.12-minimalCentOS-Stream-9x86_64✅ passed02.12.2025 16:18:2319min 44stest pipeline
RHEL9 - 3.11RHEL-9.6.0-Nightlyx86_64✅ passed02.12.2025 16:04:4031min 5stest pipeline
RHEL9 - 3.12RHEL-9.6.0-Nightlyx86_64✅ passed02.12.2025 16:24:2929min 37stest pipeline
CentOS Stream 9 - PyTest - 3.11CentOS-Stream-9x86_64✅ passed02.12.2025 15:24:5818min 59stest pipeline
CentOS Stream 9 - PyTest - 3.12CentOS-Stream-9x86_64✅ passed02.12.2025 15:24:4918min 1stest pipeline
CentOS Stream 9 - 3.9-minimalCentOS-Stream-9x86_64✅ passed02.12.2025 15:25:0519min 3stest pipeline
CentOS Stream 9 - 3.12CentOS-Stream-9x86_64✅ passed02.12.2025 16:20:5421min 23stest pipeline
CentOS Stream 9 - 3.11CentOS-Stream-9x86_64✅ passed02.12.2025 16:14:1022min 20stest pipeline
CentOS Stream 10 - 3.12-minimalCentOS-Stream-10x86_64✅ passed02.12.2025 16:14:2917min 12stest pipeline
CentOS Stream 9 - 3.9CentOS-Stream-9x86_64✅ passed02.12.2025 15:25:1921min 59stest pipeline
Fedora - 3.13Fedora-latestx86_64✅ passed02.12.2025 15:25:1820min 13stest pipeline
RHEL9 - OpenShift 4 - 3.12-minimalRHEL-9.6.0-Nightlyx86_64✅ passed02.12.2025 15:47:2425min 56stest pipeline
RHEL10 - OpenShift 4 - 3.12-minimalRHEL-10-Nightlyx86_64✅ passed02.12.2025 15:25:2623min 60stest pipeline
RHEL8 - OpenShift 4 - 3.9RHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 15:57:4724min 9stest pipeline
RHEL9 - OpenShift 4 - 3.9RHEL-9.6.0-Nightlyx86_64✅ passed02.12.2025 15:45:3728min 30stest pipeline
RHEL9 - OpenShift 4 - 3.12RHEL-9.6.0-Nightlyx86_64✅ passed02.12.2025 16:13:5926min 17stest pipeline
RHEL9 - OpenShift 4 - 3.11RHEL-9.6.0-Nightlyx86_64✅ passed02.12.2025 16:06:2030min 24stest pipeline
RHEL8 - OpenShift 4 - 3.12-minimalRHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 16:17:3821min 3stest pipeline
RHEL8 - OpenShift 4 - 3.12RHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 16:13:5421min 36stest pipeline
RHEL8 - OpenShift 4 - 3.11RHEL-8.10.0-Nightlyx86_64✅ passed02.12.2025 16:07:5325min 36stest pipeline
CentOS Stream 9 - 3.11-minimalCentOS-Stream-9x86_64✅ passed02.12.2025 16:20:3721min 49stest pipeline
Fedora - 3.13-minimalFedora-latestx86_64✅ passed02.12.2025 16:00:1318min 4stest pipeline
CentOS Stream 10 - 3.13CentOS-Stream-10x86_64✅ passed02.12.2025 16:18:1720min 50stest pipeline
CentOS Stream 10 - PyTest - 3.13CentOS-Stream-10x86_64✅ passed02.12.2025 15:24:5617min 10stest pipeline
CentOS Stream 10 - 3.13-minimalCentOS-Stream-10x86_64✅ passed02.12.2025 16:30:0718min 29stest pipeline

@phracek
Copy link
Member Author

phracek commented Nov 11, 2025

RHEL8 failures are caused by CDN errors:

[MIRROR] libffi-devel-3.1-24.el8.x86_64.rpm: Curl error (28): Timeout was reached for https://stagecdn.redhat.com/content/dist/rhel8/8/x86_64/baseos/os/Packages/l/libffi-devel-3.1-24.el8.x86_64.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds]
[MIRROR] hunspell-en-0.20140811.1-12.el8.noarch.rpm: Curl error (56): Failure when receiving data from the peer for https://stagecdn.redhat.com/content/dist/rhel8/8/x86_64/appstream/os/Packages/h/hunspell-en-0.20140811.1-12.el8.noarch.rpm [Received HTTP code 503 from proxy after CONNECT]
[MIRROR] libtalloc-2.4.1-0.el8.x86_64.rpm: Curl error (28): Timeout was reached for https://stagecdn.redhat.com/content/dist/rhel8/8/x86_64/baseos/os/Packages/l/libtalloc-2.4.1-0.el8.x86_64.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds]
[MIRROR] openldap-devel-2.4.46-21.el8_10.x86_64.rpm: Curl error (28): Timeout was reached for https://stagecdn.redhat.com/content/dist/rhel8/8/x86_64/baseos/os/Packages/o/openldap-devel-2.4.46-21.el8_10.x86_64.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds]
[MIRROR] hunspell-en-GB-0.20140811.1-12.el8.noarch.rpm: Curl error (56): Failure when receiving data from the peer for https://stagecdn.redhat.com/content/dist/rhel8/8/x86_64/appstream/os/Packages/h/hunspell-en-GB-0.20140811.1-12.el8.noarch.rpm [Received HTTP code 503 from proxy after CONNECT]
[MIRROR] libffi-devel-3.1-24.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for https://stagecdn.redhat.com/content/dist/rhel8/8/x86_64/baseos/os/Packages/l/libffi-devel-3.1-24.el8.x86_64.rpm [Received HTTP code 503 from proxy after CONNECT]
[FAILED] libffi-devel-3.1-24.el8.x86_64.rpm: No more mirrors to try - All mirrors were already tried without success

The downloaded packages were saved in cache until the next successful t

@phracek
Copy link
Member Author

phracek commented Nov 12, 2025

[test][test-all]

@phracek
Copy link
Member Author

phracek commented Nov 13, 2025

The new container-ci-suite was released. Let's re-test it.

[test-all]

@phracek
Copy link
Member Author

phracek commented Nov 13, 2025

Python failures are:

C10S - PyTest - 3.12-minimal:

[notice] A new release of pip is available: 24.2 -> 25.3
[notice] To update, run: pip install --upgrade pip
--> 04db29fbc4be
[2/2] STEP 1/6: FROM quay.io/sclorg/python-312-minimal-c10s:3.12-minimal
[2/2] STEP 2/6: COPY --from=builder $APP_ROOT $APP_ROOT
--> 8b1aba1b0f19
[2/2] STEP 3/6: USER 0
--> 8f0c8050bef9
[2/2] STEP 4/6: RUN microdnf install -y httpd enchant
Downloading metadata...
Downloading metadata...
Downloading metadata...
error: No package matches 'enchant'
Error: building at STEP "RUN microdnf install -y httpd enchant": while running runtime: exit status 1

It caused, I guess, main image should not be pulled. C10S - 3.12 should not be exists in quay.io/sclorg/python-312-c10s. RHEL10 does not exists.

Fedora PyTest - 3.13 - test_container_basics.py::TestS2IPythonContainer::test_dockerfiles[Dockerfile.tpl] INFO:Image name to test: quay.io/fedora/python3-313:3.13

Downloading setuptools-80.9.0-py3-none-any.whl (1.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 428.5 MB/s eta 0:00:00
Building wheels for collected packages: psycopg2-binary
  Building wheel for psycopg2-binary (pyproject.toml): started
  Building wheel for psycopg2-binary (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error
  
  × Building wheel for psycopg2-binary (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [115 lines of output]
      /tmp/pip-build-env-jyymjeu8/overlay/lib/python3.13/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
      !!
      
              ********************************************************************************
              Please consider removing the following classifiers in favor of a SPDX license expression:
      
              License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL)
      
              See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
              ********************************************************************************
      
      !!
        self._finalize_license_expression()
      running bdist_wheel
      running build
      running build_py
      creating build/lib.linux-x86_64-cpython-313/psycopg2

RHEL8 - 3.11 and RHEL9-PyTest-3.11

[notice] To update, run: python3.11 -m pip install --upgrade pip
---> Installing dependencies via pipenv ...
Courtesy Notice:
Pipenv found itself running within a virtual environment,  so it will 
automatically use that environment, instead of  creating its own for any 
project. You can set
PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and 
create  its own instead.
You can set PIPENV_VERBOSITY=-1 to suppress this warning.
Installing dependencies from Pipfile.lock (63c7a0)...
Looking in indexes: https://pypi.python.org/simple
Obtaining file:///opt/app-root/src (from -r 
/tmp/pipenv-48jubp3a-requirements/pipenv-7im0x6ci-reqs.txt (line 1))
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Checking if build backend supports build_editable: started
  Checking if build backend supports build_editable: finished with status 'done'
ERROR: Exception:
Traceback (most recent call last):
  File 
"/opt/app-root/src/.local/venvs/pipenv/lib/python3.11/site-packages/pipenv/patch
ed/pip/_internal/cli/base_command.py", line 105, in _run_wrapper
    status = _inner_run()
             ^^^^^^^^^^^^
  File 
"/opt/app-root/src/.local/venvs/pipenv/lib/python3.11/site-packages/pipenv/patch
ed/pip/_internal/cli/base_command.py", line 96, in _inner_run
    return self.run(options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File 
"/opt/app-root/src/.local/venvs/pipenv/lib/python3.11/site-packages/pipenv/patch
ed/pip/_internal/cli/req_command.py", line 68, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File

@frenzymadness
Copy link
Member

frenzymadness commented Nov 14, 2025

It caused, I guess, main image should not be pulled. C10S - 3.12 should not be exists in quay.io/sclorg/python-312-c10s. RHEL10 does not exists.

I don't understand this sentence, sorry. We have dockerfiles here for 3.12 and 3.12-minimal based on C10S.

Fedora PyTest - 3.13 - test_container_basics.py::TestS2IPythonContainer::test_dockerfiles[Dockerfile.tpl] INFO:Image name to test: quay.io/fedora/python3-313:3.13

The version of psycopg2-binary I see in the logs indicates that this test used django-ex branch 3.2.x and that's incorrect. New Pythons like 3.13 should use Django version/branch 4.2.x that has newer psycopg2-binary dependency and that new version provides wheels for Python 3.13.

RHEL8 - 3.11 and RHEL9-PyTest-3.11

I need more time to investigate the last one. I'm not able to reproduce locally so I need to build the container image first.

@phracek
Rename test_* python files for OpenShift to test_ocp.
5a00220 
Also VARS.* variables are used during the migration.

Signed-off-by: Petr "Stone" Hracek <[email protected]>
@phracek
Introduce PyTest suite migrated from test/run bash suite.
bc50773 
Migration matrix is:

`build_s2i_app` is fixture that takes `Path` test-app from `conftest.py`
file.

test_container_application.py tests all test-app directories.

test_container_basics.py tests only basics w/o application.
It also tests test_dockerfile and test_minimal_dockerfile

Signed-off-by: Petr "Stone" Hracek <[email protected]>
@phracek
Add dist-gen generated files
8ea1dec 
Signed-off-by: Petr "Stone" Hracek <[email protected]>
@phracek
Run all tests and not only test_container_basics.py
8249fc6 
Add distgen generated files

Signed-off-by: Petr "Stone" Hracek <[email protected]>
@phracek
Add Unstable app to PyTest suite.
df1b3e2 
Fix proper branch in `test_dockerfiles` function.

Add test suite to Class and build application once.

Signed-off-by: Petr "Stone" Hracek <[email protected]>
@phracek
Copy link
Member Author

phracek commented Nov 18, 2025

I have fixed some issues. Let's re-run tests again
[test-all]

frenzymadness
frenzymadness reviewed Nov 21, 2025
View reviewed changes
Copy link
Member

@frenzymadness frenzymadness left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good overall. Just a few nitpicks.

@frenzymadness
Copy link
Member

frenzymadness commented Nov 21, 2025

Let me know if you want me to take a look at the results of the failed tests.

@phracek
remove MINIMAL_WEB_APPS, we can add them later on, in case needed
2b7a9b7 
Use default branch 4.2.X for all versions except 3.6

Signed-off-by: Petr "Stone" Hracek <[email protected]>
@phracek
Copy link
Member Author

phracek commented Nov 26, 2025

Let's test it again

[test-pytest]

@phracek
Skip tests for full container image in case
4638046 
minimal one does not have it.

Signed-off-by: Petr "Stone" Hracek <[email protected]>
@phracek
Copy link
Member Author

phracek commented Nov 27, 2025

[test-pytest]

@frenzymadness
Copy link
Member

frenzymadness commented Dec 1, 2025

RHEL 10 - 3.12-minimal

Pulling image ubi10-minimal:latest failed. But it works fine for me. Restart might fix it.

C10S - 3.12-minimal

You can remove enchant from src/test/from-dockerfile/mod_wsgi.Dockerfile.tpl. It seems it works without it now, and the package is no longer available in UBI 10.

@frenzymadness frenzymadness mentioned this pull request Dec 1, 2025
Open
@phracek
Remove enchant package from
66c3215 
`src/test/from-dockerfile/mod_wsgi.Dockerfile.tpl`.

It is not needed and the tests works well.

Signed-off-by: Petr "Stone" Hracek <[email protected]>
@phracek
Copy link
Member Author

phracek commented Dec 2, 2025

[test-all]

@phracek
Copy link
Member Author

phracek commented Dec 3, 2025

RHEL8 - 3.12-minimal there is an issue with ethel. Infrastructure issue. I dont' know when it will be solve it.

@phracek phracek merged commit 085aca5 into sclorg:master Dec 4, 2025
64 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@frenzymadness frenzymadness frenzymadness left review comments

Assignees

No one assigned

Labels

ready for review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@phracek @frenzymadness