Skip to content

nginxinc/nginx-unprivileged:stable-alpine errors when starting #302

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
millerick opened this issue Apr 25, 2025 · 14 comments · Fixed by #303
Closed

nginxinc/nginx-unprivileged:stable-alpine errors when starting #302

millerick opened this issue Apr 25, 2025 · 14 comments · Fixed by #303
Labels
bug Something isn't working

Comments

@millerick
Copy link

Bug Overview

After the latest update was pushed, trying to start nginx inside of the image errors.

/ $ nginx
2025/04/25 23:10:00 [notice] 14#14: using the "epoll" event method
2025/04/25 23:10:00 [notice] 14#14: nginx/1.28.0
2025/04/25 23:10:00 [notice] 14#14: built by gcc 14.2.0 (Alpine 14.2.0)
2025/04/25 23:10:00 [notice] 14#14: OS: Linux 6.13.7-orbstack-00283-g9d1400e7e9c6
2025/04/25 23:10:00 [notice] 14#14: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2025/04/25 23:10:00 [emerg] 15#15: open() "/run/nginx.pid" failed (13: Permission denied)
nginx: [emerg] open() "/run/nginx.pid" failed (13: Permission denied)

Expected Behavior

Running nginx is able to start like it did previously in nginxinc/nginx-unprivileged:stable-alpine3.20

/ $ nginx
2025/04/25 23:13:39 [notice] 13#13: using the "epoll" event method
2025/04/25 23:13:39 [notice] 13#13: nginx/1.26.3
2025/04/25 23:13:39 [notice] 13#13: built by gcc 13.2.1 20240309 (Alpine 13.2.1_git20240309)
2025/04/25 23:13:39 [notice] 13#13: OS: Linux 6.13.7-orbstack-00283-g9d1400e7e9c6
2025/04/25 23:13:39 [notice] 13#13: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2025/04/25 23:13:39 [notice] 14#14: start worker processes
2025/04/25 23:13:39 [notice] 14#14: start worker process 15
/ $ 2025/04/25 23:13:39 [notice] 14#14: start worker process 16
2025/04/25 23:13:39 [notice] 14#14: start worker process 17
2025/04/25 23:13:39 [notice] 14#14: start worker process 18
2025/04/25 23:13:39 [notice] 14#14: start worker process 19
2025/04/25 23:13:39 [notice] 14#14: start worker process 20
2025/04/25 23:13:39 [notice] 14#14: start worker process 21
2025/04/25 23:13:39 [notice] 14#14: start worker process 22
2025/04/25 23:13:39 [notice] 14#14: start worker process 23
2025/04/25 23:13:39 [notice] 14#14: start worker process 24
2025/04/25 23:13:39 [notice] 14#14: start worker process 25
2025/04/25 23:13:39 [notice] 14#14: start worker process 26
2025/04/25 23:13:39 [notice] 14#14: start worker process 27
2025/04/25 23:13:39 [notice] 14#14: start worker process 28

Steps to Reproduce the Bug

Dockfile contents

FROM nginxinc/nginx-unprivileged:stable-alpine

ENTRYPOINT ["nginx"]

Build and run it with docker run -it $(docker build -q .). Observe that the same does not happen if nginxinc/nginx-unprivileged:stable-alpine3.20 is used, but does happen in nginxinc/nginx-unprivileged:stable-alpine3.21 and nginxinc/nginx-unprivileged:stable-alpine.

Environment Details

Docker version 27.5.1, build 9f9e405
This is happening on my Mac laptop, as well as in the build system in all of our environments.

Additional Context

No response

@millerick millerick added the bug Something isn't working label Apr 25, 2025
@millerick
Copy link
Author

Between nginxinc/nginx-unprivileged:stable-alpine3.20 and nginxinc/nginx-unprivileged:stable-alpine3.21 it looks like /etc/nginx/nginx.conf was changed from

pid        /tmp/nginx.pid;

to

pid        /run/nginx.pid;

@millerick
Copy link
Author

The problem comes from here:

&& sed -i 's,/var/run/nginx.pid,/tmp/nginx.pid,' /etc/nginx/nginx.conf \

It looks like the underlying location changed from /var/run/nginx.pid to /run/nginx.pid and now the replacement no longer works.

I will leave it to the maintainers to decide if this is a regression in this repository or if it is a regression in nginx itself and fix the issue in the appropriate location.

@millerick
Copy link
Author

PR that moved the default location: nginx/pkg-oss#26

@rhee876527
Copy link

For what it's worth, I was able to rollback to version: nginxinc/nginx-unprivileged:1.27.4-alpine3.21-slim which was released 5 days ago so this error appears to be more recent and may not be related to alpine 3.21 versions.

@jngrb
Copy link
Contributor

jngrb commented Apr 26, 2025

In can confirm that the issue is in:

  • docker.io/nginxinc/nginx-unprivileged:1.27.5
  • docker.io/nginxinc/nginx-unprivileged:1.27.5-alpine

(both from yesterday)

while

  • docker.io/nginxinc/nginx-unprivileged:1.27.4
  • docker.io/nginxinc/nginx-unprivileged:1.27.4-alpine
  • docker.io/nginxinc/nginx-unprivileged:1.26.3
  • docker.io/nginxinc/nginx-unprivileged:1.26.3-alpine

all still work (from 5 and 4 days ago, respectively)

So it seems to be related to the 1.27.4 -> 1.27.5 switch matching with millerick's suggestion that PR nginx/pkg-oss#26 is to blame

@jngrb
Copy link
Contributor

jngrb commented Apr 26, 2025

Created a PR #303 with a proposal for a fix

May I please ask why such a critical problem (containers don't even start under any condition) is not noticed in a CI/CD prior to the release to docker.io and other public image hubs?

There some mentioning about pipeline problems in #301 though...

@maxweisspoker
Copy link

May I please ask why such a critical problem (containers don't even start under any condition) is not noticed in a CI/CD prior to the release to docker.io and other public image hubs?

I'm curious about this as well. Hit my cluster an hour ago. Extremely frustrating that this is getting released under a "stable" tag.

@boxexchanger
Copy link

+1

@ewancoder
Copy link

Yes same here, was trying to debug a build pipeline for hours and finally figured it's a bug. Clean docker run nginxinc/nginx-unprivileged:stable-alpine crashes on two different machines with /run/nginx.pid' Permission denied.

@hamzanajeebkhaan
Copy link

+1

@alessfg
Copy link
Member

alessfg commented Apr 27, 2025

I will have new images released asap. The change in the default PID path was not communicated and is not included in any of our release notes so I totally missed it. It only affects the 1.27.5 and 1.28.0 tags as @jngrb detailed.

Apologies for any broken pipelines. The CI/CD pipeline for these images needs some work to catch these issue beforehand but these images are maintained on a best effort basis and my current bandwidth is very limited. PRs are more than welcome and appreciated!

@maxweisspoker
Copy link

but these images are maintained on a best effort basis and my current bandwidth is very limited. PRs are more than welcome and appreciated!

Holy smokes, I had no idea this was a community effort.. Thanks so much for the work you do, and I will absolutely adjust my expectations!

@shkrid
Copy link

shkrid commented Apr 27, 2025

@alessfg nginxinc/nginx-unprivileged:stable-alpine still outdated on docker hub.

TAG
[stable-alpine](https://hub.docker.com/layers/nginxinc/nginx-unprivileged/stable-alpine/images/sha256-4f3cae7411031089211d2681070dc3cc5d6d22da0fe898dfcd7efc5802463f3c)
Last pushed 2 days by [alessfg](https://hub.docker.com/u/alessfg)

@alessfg
Copy link
Member

alessfg commented Apr 27, 2025

Yep the image failed to build. I've restarted the pipeline so hopefully in ~4-5 hours the newest images should be live.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
9 participants