Why does docker swarm containers always say up to date?

[renetsk]

Hello,
just trying to use Watchtower. Have created stack in portainer:

version: "3.8"
services:
  watchtower:
    image: containrrr/watchtower:latest
    container_name: watchtower
    environment:
      WATCHTOWER_DEBUG: "true"
      WATCHTOWER_MONITOR_ONLY: "true"
      WATCHTOWER_POLL_INTERVAL: 3600
      TZ: Europe/Bratislava
      WATCHTOWER_NOTIFICATIONS: email
      WATCHTOWER_NOTIFICATION_EMAIL_FROM: [email protected]
      WATCHTOWER_NOTIFICATION_EMAIL_TO: [email protected]
      # you have to use a network alias here, if you use your own certificate
      WATCHTOWER_NOTIFICATION_EMAIL_SERVER: smtp.email.com
      WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PORT: 465
      WATCHTOWER_NOTIFICATION_EMAIL_SERVER_USER: [email protected]
      WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PASSWORD: mypassword
      WATCHTOWER_NOTIFICATION_EMAIL_DELAY: 2
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

I'm getting email:

Watchtower 1.5.3
Using notifications: smtp
Checking all containers (except explicitly disabled with label)
Scheduling first run: 2023-05-18 10:45:09 +0200 CEST
Note that the first check will be performed in 59 minutes, 59 seconds

Could not do a head request for "containrrr/watchtower:latest@sha256:*****", falling back to regular pull.
Reason: registry responded to head request with "404 Not Found", auth: "not present"

Could not do a head request for "plexinc/pms-docker:latest@sha256:*****", falling back to regular pull.
Reason: registry responded to head request with "404 Not Found", auth: "not present"

Could not do a head request for "linuxserver/unifi-controller:latest@sha256:*****", falling back to regular pull.
Reason: registry responded to head request with "404 Not Found", auth: "not present"

In container logs I see:

time="2023-05-18T09:42:56+02:00" level=debug msg="Trying to load authentication credentials." container=/unifi-controller_unifi.1.lcjox8etxkoclp15v5732pe6e image="linuxserver/unifi-controller:latest@sha256:a424e8f7b523e3f0d903b8577a7608f14f4d7671178751eb69920e23dfaf092f"
time="2023-05-18T09:42:56+02:00" level=debug msg="No credentials for linuxserver found" config_file=/config.json
time="2023-05-18T09:42:56+02:00" level=debug msg="Got image name: linuxserver/unifi-controller:latest@sha256:a424e8f7b523e3f0d903b8577a7608f14f4d7671178751eb69920e23dfaf092f"
time="2023-05-18T09:42:56+02:00" level=debug msg="Checking if pull is needed" container=/unifi-controller_unifi.1.lcjox8etxkoclp15v5732pe6e image="linuxserver/unifi-controller:latest@sha256:a424e8f7b523e3f0d903b8577a7608f14f4d7671178751eb69920e23dfaf092f"
time="2023-05-18T09:42:56+02:00" level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/"
time="2023-05-18T09:42:56+02:00" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized"
time="2023-05-18T09:42:56+02:00" level=debug msg="Checking challenge header content" realm="https://auth.docker.io/token" service=registry.docker.io
time="2023-05-18T09:42:56+02:00" level=debug msg="Setting scope for auth token" image=linuxserver/unifi-controller scope="repository:linuxserver/unifi-controller:pull"
time="2023-05-18T09:42:56+02:00" level=debug msg="No credentials found."
time="2023-05-18T09:42:56+02:00" level=debug msg="Parsing image ref" host=index.docker.io image=linuxserver/unifi-controller normalized="docker.io/linuxserver/unifi-controller:latest@sha256:a424e8f7b523e3f0d903b8577a7608f14f4d7671178751eb69920e23dfaf092f" tag="latest@sha256:a424e8f7b523e3f0d903b8577a7608f14f4d7671178751eb69920e23dfaf092f"
time="2023-05-18T09:42:56+02:00" level=debug msg="Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/linuxserver/unifi-controller/manifests/latest@sha256:a424e8f7b523e3f0d903b8577a7608f14f4d7671178751eb69920e23dfaf092f"
time="2023-05-18T09:42:57+02:00" level=warning msg="Could not do a head request for \"linuxserver/unifi-controller:latest@sha256:a424e8f7b523e3f0d903b8577a7608f14f4d7671178751eb69920e23dfaf092f\", falling back to regular pull." container=/unifi-controller_unifi.1.lcjox8etxkoclp15v5732pe6e image="linuxserver/unifi-controller:latest@sha256:a424e8f7b523e3f0d903b8577a7608f14f4d7671178751eb69920e23dfaf092f"
time="2023-05-18T09:42:57+02:00" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/unifi-controller_unifi.1.lcjox8etxkoclp15v5732pe6e image="linuxserver/unifi-controller:latest@sha256:a424e8f7b523e3f0d903b8577a7608f14f4d7671178751eb69920e23dfaf092f"
time="2023-05-18T09:42:57+02:00" level=debug msg="Pulling image" container=/unifi-controller_unifi.1.lcjox8etxkoclp15v5732pe6e image="linuxserver/unifi-controller:latest@sha256:a424e8f7b523e3f0d903b8577a7608f14f4d7671178751eb69920e23dfaf092f"
time="2023-05-18T09:42:57+02:00" level=debug msg="No new images found for /unifi-controller_unifi.1.lcjox8etxkoclp15v5732pe6e"
time="2023-05-18T09:42:57+02:00" level=debug msg="Trying to load authentication credentials." container=/vaultwarden_vaultwarden.1.v5uain2ef5zbrrs1l8rmwqpiu image="vaultwarden/server:latest@sha256:4e56e28a265035b5f8920a01d042aac9f68f1876cb260e799b94363dfbbdb9bf"
time="2023-05-18T09:42:57+02:00" level=debug msg="No credentials for vaultwarden found" config_file=/config.json
time="2023-05-18T09:42:57+02:00" level=debug msg="Got image name: vaultwarden/server:latest@sha256:4e56e28a265035b5f8920a01d042aac9f68f1876cb260e799b94363dfbbdb9bf"
time="2023-05-18T09:42:57+02:00" level=debug msg="Checking if pull is needed" container=/vaultwarden_vaultwarden.1.v5uain2ef5zbrrs1l8rmwqpiu image="vaultwarden/server:latest@sha256:4e56e28a265035b5f8920a01d042aac9f68f1876cb260e799b94363dfbbdb9bf"
time="2023-05-18T09:42:57+02:00" level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/"
time="2023-05-18T09:42:58+02:00" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized"
time="2023-05-18T09:42:58+02:00" level=debug msg="Checking challenge header content" realm="https://auth.docker.io/token" service=registry.docker.io
time="2023-05-18T09:42:58+02:00" level=debug msg="Setting scope for auth token" image=vaultwarden/server scope="repository:vaultwarden/server:pull"
time="2023-05-18T09:42:58+02:00" level=debug msg="No credentials found."
time="2023-05-18T09:42:58+02:00" level=debug msg="Parsing image ref" host=index.docker.io image=vaultwarden/server normalized="docker.io/vaultwarden/server:latest@sha256:4e56e28a265035b5f8920a01d042aac9f68f1876cb260e799b94363dfbbdb9bf" tag="latest@sha256:4e56e28a265035b5f8920a01d042aac9f68f1876cb260e799b94363dfbbdb9bf"
time="2023-05-18T09:42:58+02:00" level=debug msg="Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/vaultwarden/server/manifests/latest@sha256:4e56e28a265035b5f8920a01d042aac9f68f1876cb260e799b94363dfbbdb9bf"
time="2023-05-18T09:42:58+02:00" level=warning msg="Could not do a head request for \"vaultwarden/server:latest@sha256:4e56e28a265035b5f8920a01d042aac9f68f1876cb260e799b94363dfbbdb9bf\", falling back to regular pull." container=/vaultwarden_vaultwarden.1.v5uain2ef5zbrrs1l8rmwqpiu image="vaultwarden/server:latest@sha256:4e56e28a265035b5f8920a01d042aac9f68f1876cb260e799b94363dfbbdb9bf"
time="2023-05-18T09:42:58+02:00" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/vaultwarden_vaultwarden.1.v5uain2ef5zbrrs1l8rmwqpiu image="vaultwarden/server:latest@sha256:4e56e28a265035b5f8920a01d042aac9f68f1876cb260e799b94363dfbbdb9bf"
time="2023-05-18T09:42:58+02:00" level=debug msg="Pulling image" container=/vaultwarden_vaultwarden.1.v5uain2ef5zbrrs1l8rmwqpiu image="vaultwarden/server:latest@sha256:4e56e28a265035b5f8920a01d042aac9f68f1876cb260e799b94363dfbbdb9bf"
time="2023-05-18T09:42:59+02:00" level=debug msg="No new images found for /vaultwarden_vaultwarden.1.v5uain2ef5zbrrs1l8rmwqpiu"
time="2023-05-18T09:42:59+02:00" level=debug msg="Trying to load authentication credentials." container=/uptime-kuma_uptime-kuma.1.qsudmgkhsykddh8vhk7lmpmmf image="louislam/uptime-kuma:latest@sha256:1630eb7859c5825a1bc3fcbea9467ab3c9c2ef0d98a9f5f0ab0aec9791c027e8"
time="2023-05-18T09:42:59+02:00" level=debug msg="No credentials for louislam found" config_file=/config.json
time="2023-05-18T09:42:59+02:00" level=debug msg="Got image name: louislam/uptime-kuma:latest@sha256:1630eb7859c5825a1bc3fcbea9467ab3c9c2ef0d98a9f5f0ab0aec9791c027e8"
time="2023-05-18T09:42:59+02:00" level=debug msg="Checking if pull is needed" container=/uptime-kuma_uptime-kuma.1.qsudmgkhsykddh8vhk7lmpmmf image="louislam/uptime-kuma:latest@sha256:1630eb7859c5825a1bc3fcbea9467ab3c9c2ef0d98a9f5f0ab0aec9791c027e8"
time="2023-05-18T09:42:59+02:00" level=debug msg="Building challenge URL" URL="https://index.docker.io/v2/"
time="2023-05-18T09:42:59+02:00" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized"
time="2023-05-18T09:42:59+02:00" level=debug msg="Checking challenge header content" realm="https://auth.docker.io/token" service=registry.docker.io
time="2023-05-18T09:42:59+02:00" level=debug msg="Setting scope for auth token" image=louislam/uptime-kuma scope="repository:louislam/uptime-kuma:pull"
time="2023-05-18T09:42:59+02:00" level=debug msg="No credentials found."
time="2023-05-18T09:42:59+02:00" level=debug msg="Parsing image ref" host=index.docker.io image=louislam/uptime-kuma normalized="docker.io/louislam/uptime-kuma:latest@sha256:1630eb7859c5825a1bc3fcbea9467ab3c9c2ef0d98a9f5f0ab0aec9791c027e8" tag="latest@sha256:1630eb7859c5825a1bc3fcbea9467ab3c9c2ef0d98a9f5f0ab0aec9791c027e8"
time="2023-05-18T09:42:59+02:00" level=debug msg="Doing a HEAD request to fetch a digest" url="https://index.docker.io/v2/louislam/uptime-kuma/manifests/latest@sha256:1630eb7859c5825a1bc3fcbea9467ab3c9c2ef0d98a9f5f0ab0aec9791c027e8"
time="2023-05-18T09:42:59+02:00" level=warning msg="Could not do a head request for \"louislam/uptime-kuma:latest@sha256:1630eb7859c5825a1bc3fcbea9467ab3c9c2ef0d98a9f5f0ab0aec9791c027e8\", falling back to regular pull." container=/uptime-kuma_uptime-kuma.1.qsudmgkhsykddh8vhk7lmpmmf image="louislam/uptime-kuma:latest@sha256:1630eb7859c5825a1bc3fcbea9467ab3c9c2ef0d98a9f5f0ab0aec9791c027e8"
time="2023-05-18T09:42:59+02:00" level=warning msg="Reason: registry responded to head request with \"404 Not Found\", auth: \"not present\"" container=/uptime-kuma_uptime-kuma.1.qsudmgkhsykddh8vhk7lmpmmf image="louislam/uptime-kuma:latest@sha256:1630eb7859c5825a1bc3fcbea9467ab3c9c2ef0d98a9f5f0ab0aec9791c027e8"
time="2023-05-18T09:42:59+02:00" level=debug msg="Pulling image" container=/uptime-kuma_uptime-kuma.1.qsudmgkhsykddh8vhk7lmpmmf image="louislam/uptime-kuma:latest@sha256:1630eb7859c5825a1bc3fcbea9467ab3c9c2ef0d98a9f5f0ab0aec9791c027e8"
time="2023-05-18T09:43:00+02:00" level=debug msg="No new images found for /uptime-kuma_uptime-kuma.1.qsudmgkhsykddh8vhk7lmpmmf"
time="2023-05-18T09:43:04+02:00" level=info msg="Session done" Failed=0 Scanned=16 Updated=0 notify=no
time="2023-05-18T09:43:04+02:00" level=debug msg="Scheduled next run: 2023-05-18 09:43:45 +0200 CEST"

Any idea what is wrong please?

[piksel]

Your images are pinned to hashes, which is basically saying "don't update this image" (you are asking the repository "what is the latest version of the image with this hash", which can never result in a new image (since that would have another hash). Not sure what is adding the pins though. Perhaps you are using docker swarm?
I really thought we had added a specialized warning message for this, but I guess not.

[renetsk]

yes I'm using docker swarm. So what can I do now?

[piksel]

If you are using Docker Swarm, try using shepherd instead. It was purposely made for updating swarms and doesn't need to do all the shenanigans that watchtower does to make it work with regular docker containers.