Skip to content

Use the binary version of psycopg for development #2212

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 1, 2025
+7 −13
Merged

Use the binary version of psycopg for development #2212

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 4 additions & 12 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

# install deb packages
# PostgreSQL dependencies are needed for dbshell and backup process
RUN apt-get update \
&& apt-get install --assume-yes --no-install-recommends \
gettext \
Expand All @@ -23,23 +24,14 @@ RUN apt-get update \
&& apt-get distclean

ARG REQ_FILE=requirements/prod.txt
ARG BUILD_DEPENDENCIES="g++ gcc libc6-dev libpq-dev zlib1g-dev"

# install python dependencies
COPY ./requirements ./requirements
RUN apt-get update \
&& apt-get install --assume-yes --no-install-recommends \
g++ \
gcc \
libc6-dev \
libpq-dev \
zlib1g-dev \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If finally we're going to use psycopg[binary] check if this depenencies are not needed to the compilation of other dependencies (eg: pillow).

If not, and we don't need any more of these system dependencies to compile python packages, I'll invite you to get rid of the whole section of installing debian packages (also compilers), and removing an leaving only the pip install line.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gcc and g++ are required for libsass. I looked for an alternative that doesn't require compiling but couldn't find one yet.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is libsass requiring both gcc and g++ ?
Since you studied it, can we open a PR for the libsass repo to ask a wheel for recent python versions?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or we can find an alternate package to replace it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll check this again.

https://github.com/sass/libsass-python says

Provides prebuilt wheel binaries for Linux, Windows, and Mac.

but our build fails with

77.32   × Building wheel for libsass (pyproject.toml) did not run successfully.
...
77.32       error: command 'gcc' failed: No such file or directory

when those dependencies are not installed.

This feels more a like bug, rather than a missing feature in the libsass package.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @dvarrazzo, thank you for joining this conversation and for the in-depth discussion on psycopg/psycopg#1156!

To clarify one point, with this PR, we segmented the requirements so we can use psycopg[binary] for local development (to simplify the local setup process) and psycopg[c] in production, as this is how I interpreted the recommendation in the local installation instructions:

This is the preferred way to install Psycopg for a production site.

Could you clarify if psycopg[binary] is also intended for use in production environments? We control the environment (currently python:3.12-slim-trixie, plus pgdg for the correct postgresql-client version if that matters), but that will of course change as we upgrade Python & Debian releases over time.

Thanks so much for your feedback!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

About the problem with libsass, I created an issue on their repo, and it got closed as "completed" but without a fix.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

libsass follow up: #2248 (comment)

Copy link

@dvarrazzo dvarrazzo Oct 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello, sorry for the radio silence, I'm crushed by my backlog...

The reason for declaring [c] preferred is mostly related to release upgrades: updating the system libpq or openssl [c] will use the updated libraries, [binary] will not and a new release of Psycopg would be required.

Performance and stability-wise I don't expect differences.

If your service is controlled by a grumpy, customarily bearded, sysadmin who wants to have control on every system library, using [c] you will use the system ones; using [binary] you will get whatever packaged in the psycopg-binary build.

Copy link
Member Author

@ulgens ulgens Oct 11, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason for declaring [c] preferred is mostly related to release upgrades: updating the system libpq or openssl [c] will use the updated libraries

If your service is controlled by a grumpy, customarily bearded, sysadmin who wants to have control on every system library, using [c] you will use the system ones

Performance and stability-wise I don't expect differences.

@django/django-website My understanding is unless we need to manage libpq and openssl versions, [c] doesn't seem necessary. Do we still want to keep the current config or would we like to proceed with simplifaction / cleaning?

&& apt-get install --assume-yes --no-install-recommends ${BUILD_DEPENDENCIES} \
&& python3 -m pip install --no-cache-dir -r ${REQ_FILE} \
&& apt-get purge --assume-yes --auto-remove \
g++ \
gcc \
libc6-dev \
libpq-dev \
zlib1g-dev \
&& apt-get purge --assume-yes --auto-remove ${BUILD_DEPENDENCIES} \
&& apt-get distclean

# copy project
Expand Down
1 change: 1 addition & 0 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ services:
dockerfile: Dockerfile
args:
- REQ_FILE=requirements/tests.txt
- BUILD_DEPENDENCIES=g++ gcc
entrypoint: ./docker-entrypoint.dev.sh
command: python manage.py runserver 0.0.0.0:8000
volumes:
Expand Down
1 change: 0 additions & 1 deletion requirements/common.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ Jinja2==3.1.6
libsass==0.23.0
Markdown==3.9
Pillow==11.3.0
psycopg[c]==3.2.9
Pygments==2.19.2
pykismet3==0.1.1
requests==2.32.5
Expand Down
1 change: 1 addition & 0 deletions requirements/dev.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
-r common.txt
django-debug-toolbar==6.0.0
pre-commit~=4.3.0
psycopg[binary]==3.2.10
watchdog==6.0.0
1 change: 1 addition & 0 deletions requirements/prod.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
-r common.txt
gunicorn==23.0.0
psycopg[c]==3.2.10
redis==6.4.0
sentry-sdk==2.38.0