Skip to content

Bug: Database fields not escaped when building URL #547

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
LyndonFan opened this issue Apr 17, 2024 · 8 comments · Fixed by #549
Closed

Bug: Database fields not escaped when building URL #547

LyndonFan opened this issue Apr 17, 2024 · 8 comments · Fixed by #549

Comments

@LyndonFan
Copy link
Contributor

Describe the bug

The get_connection_url function for database containers does not escape "%" properly.

I was testing some code at work which connected to a database (MySQL), and noticed it was unable to do so. I found out it was because the password contained a "%" and we created the database url with f-strings like below. This meant the url got screwed up and the connection failed.

f"mysql+pymysql://{username}:{password}@{host}:{port}/{database}

While I have only tested with MySql, from going through the repo, I suspect this could affect most (if not all) database containers, as the same method appears to be used in the library.

To Reproduce

See the simplified version of my test code.

# test_create_user.py
import pytest
from sqlalchemy import create_engine
from testcontainers.mysql import MySqlContainer

mock_secrets = {
    "root_username": "root",
    "root_password": "root%1234",
}

mysql = MySqlContainer(
    "mysql:8.0.27",
    username=mock_secrets["root_username"],
    password=mock_secrets["root_password"],
)

@pytest.fixture(scope="module", autouse=True)
def mysql_container(request):
    mysql.start()
    def cleanup():
        mysql.stop()

    request.addfinalizer(cleanup)
    return mysql

def test_create_user():
    root_url = mysql.get_connection_url() # just need to make sure mysql is used
  • Expected behaviour: sql container spins up, and the test passes
  • Actual behaviour: the test fails at mysql.start(), as it is unable to connect to the container.
Full error logs 
(testcontainers-issue-py3.11) MacBook-Pro-9:testcontainers-issue lyndonf$ pytest .
======================================================= test session starts ========================================================
platform darwin -- Python 3.11.2, pytest-8.1.1, pluggy-1.4.0
rootdir: /Users/lyndonf/Desktop/testcontainers-issue
configfile: pyproject.toml
collected 1 item

test_create_user.py E [100%]

====================================================== ERRORS ======================================================
******************\_\_\_\_****************** ERROR at setup of test_create_user ******************\_\_\_\_******************

request = <SubRequest 'mysql_container' for <Function test_create_user>>

    @pytest.fixture(scope="module", autouse=True)
    def mysql_container(request):

>       mysql.start()

test_create_user.py:22:

---

.venv/lib/python3.11/site-packages/testcontainers/core/generic.py:71: in start
self.\_connect()

---

wrapped = <bound method DbContainer.\_connect of <testcontainers.mysql.MySqlContainer object at 0x10d17c350>>
instance = <testcontainers.mysql.MySqlContainer object at 0x10d17c350>, args = (), kwargs = {}

    @wrapt.decorator
    def wrapper(wrapped: Callable, instance: Any, args: list, kwargs: dict) -> Any:
        from testcontainers.core.container import DockerContainer

        if isinstance(instance, DockerContainer):
            logger.info("Waiting for container %s with image %s to be ready ...", instance._container, instance.image)
        else:
            logger.info("Waiting for %s to be ready ...", instance)

        exception = None
        for attempt_no in range(config.max_tries):
            try:
                return wrapped(*args, **kwargs)
            except transient_exceptions as e:
                logger.debug(
                    f"Connection attempt '{attempt_no + 1}' of '{config.max_tries + 1}' "
                    f"failed: {traceback.format_exc()}"
                )
                time.sleep(config.sleep_time)
                exception = e

>       raise TimeoutError(

            f"Wait time ({config.timeout}s) exceeded for {wrapped.__name__}(args: {args}, kwargs: "
            f"{kwargs}). Exception: {exception}"
        )

E TimeoutError: Wait time (120s) exceeded for \_connect(args: (), kwargs: {}). Exception: (pymysql.err.OperationalError) (1045, "Access denied for user 'root'@'172.17.0.1' (using password: YES)")
E (Background on this error at: https://sqlalche.me/e/20/e3q8)

.venv/lib/python3.11/site-packages/testcontainers/core/waiting_utils.py:67: TimeoutError
---------------------------------------------- Captured stderr setup -----------------------------------------------
Pulling image testcontainers/ryuk:0.7.0
Container started: b8f4d57ee296
Waiting for container <Container: b8f4d57ee296> with image testcontainers/ryuk:0.7.0 to be ready ...
Pulling image mysql:8.0.27
Container started: b91add5dbff1
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
------------------------------------------------ Captured log setup ------------------------------------------------
INFO testcontainers.core.container:container.py:77 Pulling image testcontainers/ryuk:0.7.0
INFO testcontainers.core.container:container.py:89 Container started: b8f4d57ee296
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b8f4d57ee296> with image testcontainers/ryuk:0.7.0 to be ready ...
INFO testcontainers.core.container:container.py:77 Pulling image mysql:8.0.27
INFO testcontainers.core.container:container.py:89 Container started: b91add5dbff1
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
INFO testcontainers.core.waiting_utils:waiting_utils.py:52 Waiting for container <Container: b91add5dbff1> with image mysql:8.0.27 to be ready ...
============================================= short test summary info ==============================================
ERROR test_create_user.py::test_create_user - TimeoutError: Wait time (120s) exceeded for \_connect(args: (), kwargs: {}). Exception: (pymysql.err.Operational...
=========================================== 1 error in 127.60s (0:02:07) ===========================================

Suggestions

  • use urllib.parse.quote to preprocess fields before putting them into the url
  • use URL from sqlalchemy to generate the url, though from previous history this option does not seem to be recommended

Runtime environment

  • Operating System: Darwin MacBook-Pro-9 22.3.0
  • Python version: 3.11.2
  • Docker Version: Docker version 20.10.17, build 100c701
  • testcontainers version: 4.3.3
Full environment details 
(testcontainers-issue-py3.11) MacBook-Pro-9:testcontainers-issue lyndonf$ uname -a
Darwin MacBook-Pro-9 22.3.0 Darwin Kernel Version 22.3.0: Mon Jan 30 20:42:11 PST 2023; root:xnu-8792.81.3~2/RELEASE_X86_64 x86_64
(testcontainers-issue-py3.11) MacBook-Pro-9:testcontainers-issue lyndonf$ python --version
Python 3.11.2
(testcontainers-issue-py3.11) MacBook-Pro-9:testcontainers-issue lyndonf$ docker --version
Docker version 20.10.17, build 100c701
(testcontainers-issue-py3.11) MacBook-Pro-9:testcontainers-issue lyndonf$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.9.1)
  compose: Docker Compose (Docker Inc., v2.10.2)
  extension: Manages Docker extensions (Docker Inc., v0.2.9)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.19.0)

Server:
 Containers: 4
  Running: 3
  Paused: 0
  Stopped: 1
 Images: 11
 Server Version: 20.10.17
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.10.124-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 3.842GiB
 Name: docker-desktop
 ID: AP6V:IK5D:CB2F:AUKG:RNID:EJ5Y:L4PD:2BR6:FR57:7FH2:SHCA:2AZB
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Live Restore Enabled: false

(testcontainers-issue-py3.11) MacBook-Pro-9:testcontainers-issue lyndonf$ pip freeze
certifi==2024.2.2
cffi==1.16.0
charset-normalizer==3.3.2
cryptography==42.0.5
docker==7.0.0
greenlet==3.0.3
idna==3.7
iniconfig==2.0.0
packaging==24.0
pluggy==1.4.0
pycparser==2.22
PyMySQL==1.1.0
pytest==8.1.1
requests==2.31.0
ruff==0.3.7
SQLAlchemy==2.0.29
testcontainers==4.3.3
typing_extensions==4.11.0
urllib3==2.2.1
wrapt==1.16.0
(testcontainers-issue-py3.11) MacBook-Pro-9:testcontainers-issue lyndonf$
@max-pfeiffer
Copy link
Contributor

max-pfeiffer commented Apr 17, 2024 •
edited
Loading

@LyndonFan I just wrote a test and tried to reproduce your issue: and yes it fails also on my machine, but not because of an character escaping problem. It fails because the mysql:8.0.27 Docker image does not support ARM CPUs.

When I run the same test with mysql:8.3.0 image, test is green.

Please check the images on Docker Hub:

Are you working on an M1, M2 or M3 MacBook? Then most likely that is the root cause of your problem.

@LyndonFan
Copy link
Contributor Author

None of the above -- it's "2.3 GHz Quad-Core Intel Core i7"

@alexanderankin
Copy link
Member

using urllib.parse.quote makes sense to me, sounds like a mergeable pull request if we had one that did that 👍

@max-pfeiffer
Copy link
Contributor

max-pfeiffer commented Apr 17, 2024 •
edited
Loading

@LyndonFan I just ran your sample code with mysql:8.3.0, test is green. Cannot reproduce.

testcontainers-python=4.4.0

@alexanderankin
Copy link
Member

alexanderankin commented Apr 17, 2024 •
edited
Loading

hmm, yeah it doesn't reproduce for me either: works with %12:

mkdir testcontainers-python-issues-547; cd $_ ; python -m venv .venv && . $_/bin/activate; pip install testcontainers sqlalchemy pymsql cryptography

cat > test.py <<EOF
import sqlalchemy
from testcontainers.mysql import MySqlContainer

with MySqlContainer(
        "mysql:8-oracle",
        root_password="root%12password",
        password="pass%12word",
) as m:
    engine = sqlalchemy.create_engine(m.get_connection_url())
    with engine.connect() as connection:
        result = connection.exec_driver_sql("select version()")
        result = result.fetchall()
        print(result)
EOF

python test.py

produces:

sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1045, "Access denied for user 'test'@'172.17.0.1' (using password: YES)")

@LyndonFan
Copy link
Contributor Author

Thanks for the quick responses!

I've updated testcontainers to 4.4.0, and updated my test to be based on David's. Note I changed the password to contain "%12", which is the same as urllib.parse.quote('\x12'), and I think that leads to the issue.

# test.py
import sqlalchemy
from testcontainers.mysql import MySqlContainer

with MySqlContainer(
    "mysql:8-oracle",
    root_password="root%password",
    password="pass%12word",
) as m:
    url = m.get_connection_url()
    print(url)
    engine = sqlalchemy.create_engine(url)
    with engine.connect() as connection:
        result = connection.exec_driver_sql("select version()")
        result = result.fetchall()
        print(result)

Running python test.py, I now get access denied: (1045, "Access denied for user 'test'@'172.17.0.1' (using password: YES)")

Full error logs (8-oracle) 
(testcontainers-issue-py3.11) MacBook-Pro-9:testcontainers-issue lyndonf$ python test.py
Pulling image testcontainers/ryuk:0.7.0
Container started: 5d3e0ab834c7
Waiting for container <Container: 5d3e0ab834c7> with image testcontainers/ryuk:0.7.0 to be ready ...
Pulling image mysql:8-oracle
Container started: 76c64d30abe8
Waiting for container <Container: 76c64d30abe8> with image mysql:8-oracle to be ready ...
mysql+pymysql://test:pass%12word@localhost:58703/test
Traceback (most recent call last):
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 146, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 3304, in raw_connection
    return self.pool.connect()
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 449, in connect
    return _ConnectionFairy._checkout(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 712, in checkout
    rec = pool._do_get()
          ^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/impl.py", line 179, in _do_get
    with util.safe_reraise():
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
    return self._create_connection()
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 390, in _create_connection
    return _ConnectionRecord(self)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 674, in __init__
    self.__connect()
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 900, in __connect
    with util.safe_reraise():
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 896, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/create.py", line 643, in connect
    return dialect.connect(*cargs, **cparams)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 617, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/connections.py", line 358, in __init__
    self.connect()
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/connections.py", line 664, in connect
    self._request_authentication()
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/connections.py", line 976, in _request_authentication
    auth_packet = _auth.caching_sha2_password_auth(self, auth_packet)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/_auth.py", line 267, in caching_sha2_password_auth
    pkt = _roundtrip(conn, data)
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/_auth.py", line 120, in _roundtrip
    pkt = conn._read_packet()
          ^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/connections.py", line 772, in _read_packet
    packet.raise_for_error()
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/protocol.py", line 221, in raise_for_error
    err.raise_mysql_exception(self._data)
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.OperationalError: (1045, "Access denied for user 'test'@'172.17.0.1' (using password: YES)")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/lyndonf/Desktop/testcontainers-issue/test.py", line 15, in <module>
    with engine.connect() as connection:
         ^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 3280, in connect
    return self._connection_cls(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 148, in __init__
    Connection._handle_dbapi_exception_noconnection(
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2444, in _handle_dbapi_exception_noconnection
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 146, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 3304, in raw_connection
    return self.pool.connect()
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 449, in connect
    return _ConnectionFairy._checkout(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 712, in checkout
    rec = pool._do_get()
          ^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/impl.py", line 179, in _do_get
    with util.safe_reraise():
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
    return self._create_connection()
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 390, in _create_connection
    return _ConnectionRecord(self)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 674, in __init__
    self.__connect()
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 900, in __connect
    with util.safe_reraise():
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 896, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/create.py", line 643, in connect
    return dialect.connect(*cargs, **cparams)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 617, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/connections.py", line 358, in __init__
    self.connect()
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/connections.py", line 664, in connect
    self._request_authentication()
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/connections.py", line 976, in _request_authentication
    auth_packet = _auth.caching_sha2_password_auth(self, auth_packet)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/_auth.py", line 267, in caching_sha2_password_auth
    pkt = _roundtrip(conn, data)
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/_auth.py", line 120, in _roundtrip
    pkt = conn._read_packet()
          ^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/connections.py", line 772, in _read_packet
    packet.raise_for_error()
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/protocol.py", line 221, in raise_for_error
    err.raise_mysql_exception(self._data)
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
    raise errorclass(errno, errval)
sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1045, "Access denied for user 'test'@'172.17.0.1' (using password: YES)")
(Background on this error at: https://sqlalche.me/e/20/e3q8)

I modified the image to be mysql:8.3.0 instead of mysql:8-oracle and re-ran the test, but still got a similar error.

Full error logs (for 8.3.0) 
(testcontainers-issue-py3.11) MacBook-Pro-9:testcontainers-issue lyndonf$ python test.py
Pulling image testcontainers/ryuk:0.7.0
Container started: c5a6220ddf6b
Waiting for container <Container: c5a6220ddf6b> with image testcontainers/ryuk:0.7.0 to be ready ...
Pulling image mysql:8.3.0
Container started: 8e64d1c48b5f
Waiting for container <Container: 8e64d1c48b5f> with image mysql:8.3.0 to be ready ...
mysql+pymysql://test:pass%12word@localhost:64340/test
Traceback (most recent call last):
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 146, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 3304, in raw_connection
    return self.pool.connect()
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 449, in connect
    return _ConnectionFairy._checkout(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 712, in checkout
    rec = pool._do_get()
          ^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/impl.py", line 179, in _do_get
    with util.safe_reraise():
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
    return self._create_connection()
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 390, in _create_connection
    return _ConnectionRecord(self)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 674, in __init__
    self.__connect()
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 900, in __connect
    with util.safe_reraise():
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 896, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/create.py", line 643, in connect
    return dialect.connect(*cargs, **cparams)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 617, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/connections.py", line 358, in __init__
    self.connect()
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/connections.py", line 664, in connect
    self._request_authentication()
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/connections.py", line 976, in _request_authentication
    auth_packet = _auth.caching_sha2_password_auth(self, auth_packet)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/_auth.py", line 267, in caching_sha2_password_auth
    pkt = _roundtrip(conn, data)
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/_auth.py", line 120, in _roundtrip
    pkt = conn._read_packet()
          ^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/connections.py", line 772, in _read_packet
    packet.raise_for_error()
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/protocol.py", line 221, in raise_for_error
    err.raise_mysql_exception(self._data)
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.OperationalError: (1045, "Access denied for user 'test'@'172.17.0.1' (using password: YES)")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/lyndonf/Desktop/testcontainers-issue/test.py", line 16, in <module>
    with engine.connect() as connection:
         ^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 3280, in connect
    return self._connection_cls(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 148, in __init__
    Connection._handle_dbapi_exception_noconnection(
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2444, in _handle_dbapi_exception_noconnection
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 146, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 3304, in raw_connection
    return self.pool.connect()
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 449, in connect
    return _ConnectionFairy._checkout(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 712, in checkout
    rec = pool._do_get()
          ^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/impl.py", line 179, in _do_get
    with util.safe_reraise():
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
    return self._create_connection()
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 390, in _create_connection
    return _ConnectionRecord(self)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 674, in __init__
    self.__connect()
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 900, in __connect
    with util.safe_reraise():
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 896, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/create.py", line 643, in connect
    return dialect.connect(*cargs, **cparams)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 617, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/connections.py", line 358, in __init__
    self.connect()
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/connections.py", line 664, in connect
    self._request_authentication()
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/connections.py", line 976, in _request_authentication
    auth_packet = _auth.caching_sha2_password_auth(self, auth_packet)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/_auth.py", line 267, in caching_sha2_password_auth
    pkt = _roundtrip(conn, data)
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/_auth.py", line 120, in _roundtrip
    pkt = conn._read_packet()
          ^^^^^^^^^^^^^^^^^^^
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/connections.py", line 772, in _read_packet
    packet.raise_for_error()
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/protocol.py", line 221, in raise_for_error
    err.raise_mysql_exception(self._data)
  File "/Users/lyndonf/Desktop/testcontainers-issue/.venv/lib/python3.11/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
    raise errorclass(errno, errval)
sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1045, "Access denied for user 'test'@'172.17.0.1' (using password: YES)")
(Background on this error at: https://sqlalche.me/e/20/e3q8)

Let me know if the above results help. I'm happy to work on a merge request to close this.

@alexanderankin
Copy link
Member

yep, reproduces with %12 in the password, im editing my previous comment

@alexanderankin
Copy link
Member

@LyndonFan just re-read your last comment and yeah if you want to open a PR that would be great! not sure when ill get to it but appreciate the offer and as above, a PR is welcome. we'd probably want to either add tests for postgres, sql server, oracle, etc as well or make this behavior opt-in or something.

@LyndonFan LyndonFan mentioned this issue Apr 20, 2024
Merged
alexanderankin pushed a commit that referenced this issue Apr 20, 2024
@LyndonFan
fix: url quote passwords (#549)
6c5d227 
**Changes**

Updated DbContainer to fix #547 by using `urllib.parse.quote`. I
referenced sqlalchemy's implementation, but have not imported the
library.

I have chosen to make this behaviour occur at all times (can't opt in /
out), as it is common, if not the standard for these urls.

**Tests**

Since DbContainer can't be tested on its own, I put the tests across
various database containers. I have pasted the below as comment in the
test files for the listed modules:

```python
# This is a feature in the generic DbContainer class
# but it can't be tested on its own
# so is tested in various database modules:
# - mysql / mariadb
# - postgresql
# - sqlserver
# - mongodb
```

Note the discussion recommended me to test with oracle, but I was unable
to spin the container up locally (even with colima), so opted to replace
it with mongodb.

Is there a template for PRs for the core library? I am unable to find
one so have opted the above format. Please let me know if I have missed
anything in this PR. Thanks!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: url quote passwords LyndonFan/testcontainers-python
3 participants
@alexanderankin @max-pfeiffer @LyndonFan