Container reports "unhealthy" since upgrade from 2.6 to 2.7

[esackbauer]

However the log files to not show any error so far, and it seems to work normally:

2024-07-13T12:06:53+02:00 INFO Settings summary:
├── HTTP client
|   └── Timeout: 10s
├── Update
|   ├── Period: 3m0s
|   └── Cooldown: 5m0s
├── Public IP fetching
|   ├── HTTP enabled: yes
|   ├── HTTP IP providers
|   |   └── all
|   ├── HTTP IPv4 providers
|   |   └── all
|   ├── HTTP IPv6 providers
|   |   └── all
|   └── DNS enabled: no
├── Resolver: use Go default resolver
├── Server
|   ├── Listening address: :8000
|   └── Root URL: /
├── Health
|   └── Server listening address: 127.0.0.1:9999
├── Paths
|   ├── Data directory: /updater/data
|   └── Config file: /updater/data/config.json
├── Backup: disabled
└── Logger
    ├── Level: info
    └── Caller: hidden
2024-07-13T12:06:54+02:00 INFO reading JSON config from file /updater/data/config.json
2024-07-13T12:06:54+02:00 INFO Found 6 settings to update records
2024-07-13T12:06:55+02:00 INFO Reading history from database: domain xxxx.net owner @ ipv4
2024-07-13T12:06:55+02:00 INFO Reading history from database: domain xxxx.net owner meet ipv4
2024-07-13T12:06:55+02:00 INFO Reading history from database: domain xxxx.net owner matrix ipv4
2024-07-13T12:06:55+02:00 INFO Reading history from database: domain yyyy.de owner @ ipv4
2024-07-13T12:06:55+02:00 INFO Reading history from database: domain zzzz.org owner @ ipv4
2024-07-13T12:06:55+02:00 INFO Reading history from database: domain zzzz.org owner mail ipv4
2024-07-13T12:06:55+02:00 INFO [http server] http server listening on [::]:8000
2024-07-13T12:06:55+02:00 INFO [backup] disabled
2024/07/13 12:17:11 "GET http://app03:8000/ HTTP/1.1" from 10.0.0.2:29330 - 200 5582B in 40.889765ms
2024/07/13 12:17:11 "GET http://app03:8000/static/styles.css HTTP/1.1" from 10.0.0.2:29330 - 200 3203B in 9.018094ms
2024/07/13 12:17:11 "GET http://app03:8000/static/favicon.svg HTTP/1.1" from 10.0.0.2:29330 - 200 3192B in 668.085µs

Any ideas where to look further?

I am running on Docker 27.x on Debian 12

[qdm12]

What health check messages do you get within the response from docker inspect ddns-updater?

[esackbauer]

            "Health": {
                "Status": "unhealthy",
                "FailingStreak": 155,
                "Log": [
                    {
                        "Start": "2024-07-13T14:37:15.295736681+02:00",
                        "End": "2024-07-13T14:37:15.354986151+02:00",
                        "ExitCode": 1,
                        "Output": "2024-07-13T14:37:15+02:00 ERROR Get \"http://127.0.0.1:9999\": dial tcp 127.0.0.1:9999: connect: connection refused\n2024-07-13T14:37:15+02:00 INFO Shutdown successful\n"
                    },
                    {
                        "Start": "2024-07-13T14:38:15.355839917+02:00",
                        "End": "2024-07-13T14:38:15.42117059+02:00",
                        "ExitCode": 1,
                        "Output": "2024-07-13T14:38:15+02:00 ERROR Get \"http://127.0.0.1:9999\": dial tcp 127.0.0.1:9999: connect: connection refused\n2024-07-13T14:38:15+02:00 INFO Shutdown successful\n"
                    },
                    {
                        "Start": "2024-07-13T14:39:15.422201677+02:00",
                        "End": "2024-07-13T14:39:15.496343011+02:00",
                        "ExitCode": 1,
                        "Output": "2024-07-13T14:39:15+02:00 ERROR Get \"http://127.0.0.1:9999\": dial tcp 127.0.0.1:9999: connect: connection refused\n2024-07-13T14:39:15+02:00 INFO Shutdown successful\n"
                    },
                    {
                        "Start": "2024-07-13T14:40:15.497211964+02:00",
                        "End": "2024-07-13T14:40:15.553622849+02:00",
                        "ExitCode": 1,
                        "Output": "2024-07-13T14:40:15+02:00 ERROR Get \"http://127.0.0.1:9999\": dial tcp 127.0.0.1:9999: connect: connection refused\n2024-07-13T14:40:15+02:00 INFO Shutdown successful\n"
                    },
                    {
                        "Start": "2024-07-13T14:41:15.55401551+02:00",
                        "End": "2024-07-13T14:41:15.625473872+02:00",
                        "ExitCode": 1,
                        "Output": "2024-07-13T14:41:15+02:00 ERROR Get \"http://127.0.0.1:9999\": dial tcp 127.0.0.1:9999: connect: connection refused\n2024-07-13T14:41:15+02:00 INFO Shutdown successful\n"
                    }

and a bit down:

            "Healthcheck": {
                "Test": [
                    "CMD",
                    "/updater/ddns-updater",
                    "healthcheck"
                ],
                "Interval": 60000000000,
                "Timeout": 5000000000,
                "StartPeriod": 10000000000,
                "Retries": 2
            },

[qdm12]

Ok, so from these messages and your logs, the healthcheck server is not running.

A new feature was added in v2.7.0 where the health server wouldn't start outside docker (like in the standalone binary program). This is done by checking if the file /updater/isdocker is present or not

ddns-updater/Dockerfile

Line 76 in 9f0ca6c

COPY --from=build --chown=${UID}:${GID} /tmp/isdocker /updater/isdocker

I tried it on my machine (arch linux with docker) and it does start the health server (logs the health server is starting).

What bind mounts do you have? 🤔

[esackbauer]

I am using Portainer, and have a volume mounted to /updater/data in the container. In that docker volume there are 2 files: config.json and updates.json

[qdm12]

Maybe try completely removing the ddns container and image and then recreating it? I have seen Portainer not updating container images correctly, maybe it didn't update the file system correctly with the /updater/isdocker file.

[esackbauer]

Now it is working again, after I recreated the container. And also needed then to use the RESOLVER_ADDRESS because some records are mapped to internal addresses from within LAN.

[qdm12]

Yeah Portainer isn't great at properly updating images/containers, I'm not sure why. Feel free to report it to its maintainers. Anyway enjoy now that it's fixed!