SeleniumBase Tests Fail on Gitpod Workspace Requiring Manual Rebuild

[pa-decarvalho]

Bug description

When launching a Gitpod workspace, SeleniumBase tests do not pass initially. A manual gp validate is required to rebuild the image and restart the workspace, after which the tests successfully run.

I use the same image on my gitlab ci to run the tests and it works without any changes.

The Dockerfile :

FROM gitpod/workspace-python-3.12:2024-11-20-08-19-55

ARG TASK_VERSION=3.40.0
ARG HADOLINT_VERSION=2.12.0

USER root

### Install dependencies ###
# hadolint ignore=DL3008
RUN apt-get update -y && apt-get upgrade -y && \
    apt-get install -y --no-install-recommends \
    fonts-liberation fonts-liberation2 \
    fonts-font-awesome fonts-ubuntu \
    fonts-terminus fonts-powerline \
    fonts-open-sans fonts-mononoki \
    fonts-roboto fonts-lato \
    libasound2 libatk-bridge2.0-0 \
    libatk1.0-0 libatspi2.0-0 libcups2 \
    libdbus-1-3 libdrm2 libgbm1 \
    libgtk-3-0 libnspr4 libnss3 \
    libu2f-udev libvulkan1 libwayland-client0 \
    libxcomposite1 libxdamage1 libxfixes3 \
    libxkbcommon0 libxrandr2 \
    sudo unzip wget xvfb curl \
    ca-certificates git bash build-essential \
    libssl-dev zlib1g-dev libbz2-dev libreadline-dev \
    libsqlite3-dev libncursesw5-dev xz-utils \
    tk-dev libxml2-dev libxmlsec1-dev libffi-dev \
    liblzma-dev docker && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

### Install Task (https://taskfile.dev/) ###
RUN wget -q -P /tmp https://github.com/go-task/task/releases/download/v$TASK_VERSION/task_linux_amd64.deb && \
    dpkg -i /tmp/task_linux_amd64.deb && \
    rm -f /tmp/task_linux_amd64.deb

### Install Hadolint (https://github.com/hadolint/hadolint) ###
RUN wget -q -P /tmp https://github.com/hadolint/hadolint/releases/download/v$HADOLINT_VERSION/hadolint-Linux-x86_64 && \
    mv /tmp/hadolint-Linux-x86_64 /usr/local/bin/hadolint && \
    chmod +x /usr/local/bin/hadolint

### Install Google Chrome (for visual tests) ###
RUN apt-get update -y && apt-get upgrade -y && \
    wget -q https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb && \
    apt-get install -y --no-install-recommends ./google-chrome-stable_current_amd64.deb && \
    rm google-chrome-stable_current_amd64.deb && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

### Install python versions (https://github.com/pyenv/pyenv) ###
RUN pyenv install 3.9 3.10 3.11 3.13 && \
    pyenv global 3.12 && \
    pip install --no-cache-dir pip==24.3.1 poetry==1.8.4

USER gitpod

If anyone has insights or suggestions to help diagnose or resolve this issue, I'm open to feedback and recommendations.

Thanks in advance for any help!

Steps to reproduce

1. Run command task test:pytest (seleniumbase tests fails)
2. Run command gp validate (rebuild the image under .gitlab/images/Dockerfile.pyenv and create new workspace)
3. Under the new workspace, run command task test:pytest and it works.

Workspace affected

mkdocs-mkdocsdsfr-jdgyezs0zj0

Expected behavior

After the workspace starts, i can run my task test:pytest command and tests works.

Example repository

https://gitpod.io#snapshot/c3a0ba9b-fb21-4f03-a8ad-048f7d134d60

Anything else?

No response

[axonasif]

What is the error you get

[pa-decarvalho]

Hello @axonasif

All my tests with seleniumbase have the same error urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

E           urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

/home/gitpod/.pyenv/versions/3.12.7/lib/python3.12/http/client.py:300: ProtocolError
================================================================================================= short test summary info =================================================================================================
FAILED tests/visual/test_breadcrumb.py::BreadcrumbTest::test_breadcrumb - urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
FAILED tests/visual/test_component_accordion.py::ComponentAccordionTest::test_component_accordion_has_not_changed - urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
FAILED tests/visual/test_component_alert.py::ComponentAlertTest::test_component_alert_has_not_changed - urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
FAILED tests/visual/test_component_badge.py::ComponentBadgeTest::test_component_badge_has_not_changed - urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
FAILED tests/visual/test_component_callout.py::ComponentCalloutTest::test_component_callout_has_not_changed - urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
FAILED tests/visual/test_followus.py::FollowusTest::test_followus_not_visible_has_not_changed - urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
FAILED tests/visual/test_followus.py::FollowusTest::test_followus_visible_has_not_changed - urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
FAILED tests/visual/test_footer.py::FooterTest::test_footer_default_has_not_changed - urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
FAILED tests/visual/test_footer.py::FooterTest::test_footer_update_has_not_changed - urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
FAILED tests/visual/test_home_page.py::HomePageTest::test_home_page_has_not_changed - urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
FAILED tests/visual/test_not_found_page.py::NotFoundPageTest::test_not_found_page_has_not_changed - urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

It's as if a dependency were missing. And when I do gp validate and open the new workspace it works

[axonasif]

It sounds like a service is supposed to be up and running on a port? But it isn't running when you initially open the workspace. If I have time, I may take a look at your snapshot but maybe you could figure it out yourself too :)

[pa-decarvalho]

Yes, when running pytest, the command launches mkdocs serve and runs some SeleniumBase tests.

As I mentioned in my the description, I don't believe this is a pytest-related issue, as the process works both on my local machine and in the GitLab CI environment using the image example I shared earlier.

Additionally, after using the gp validate command, it also works on Gitpod.