Breaking postgrest with SIG_INT (Ctrl+C) has problem with docker

[m-ickiewicz]

Environment

• PostgreSQL version: 17
• PostgREST version: 12.0.2 and also 12.2
• Operating system: Linux 6.8.0-45-generic #45~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Sep 11 15:25:05 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
• Docker version: 27.3.1, build ce12230

Description of the issue

When PostgREST is started in Docker as the main process, terminating it with
SIGINT causes Docker to exit with code 255 instead of 130 (128 + 2 for
SIGINT).

The solution is to use tini as the Docker init process (via the --init
option in docker run). However, this suggests that PostgREST itself may have
issues with handling signals or properly shutting down its threads.

I encountered this issue while working with the Hive
HAF and
Hivemind applications.

A quick way to reproduce it is by running Hive blockchain containers for the
development branches of:

• HAF
  (I used tag b7870f22).
• Hivemind
  (I used tag e75121fd).

docker network create haf_test;
docker run -d -e PG_ACCESS="host haf_block_log all 0.0.0.0/0 trust" --network=haf_test --name=test_haf registry.gitlab.syncad.com/hive/haf/instance:b7870f22 --stop-at-block=1000;
sleep 30; # wait for syncing 1000 blocks
docker run --rm --name=hivemind_test --network=haf_test registry.gitlab.syncad.com/hive/hivemind/instance:e75121fd setup --with-reptracker  --database-admin-url=postgresql://haf_admin@test_haf/haf_block_log;
docker run --rm --name=hivemind_test --network=haf_test registry.gitlab.syncad.com/hive/hivemind/instance:e75121fd sync --test-max-block=979  --postgres-url=postgresql://hivemind@test_haf/haf_block_log;
docker run --rm --name=hivemind_test --network=haf_test --entrypoint=postgrest -e PGRST_DB_URI=postgresql://hivemind@test_haf:5432/haf_block_log registry.gitlab.syncad.com/hive/hivemind/instance:e75121fd

After executing the code above, you should see PostgREST running. Now, break
it with Ctrl+C and check echo $?. The exit code is 255, not 130.

I suppose the problem occurs in a much simpler setup, but to be sure you can
repeat it, I wanted to share my environment with you.

Hivemind issue connected with the problem: https://gitlab.syncad.com/hive/hivemind/-/issues/277

[wolfgangwalther]

When PostgREST is started in Docker as the main process, terminating it with
SIGINT causes Docker to exit with code 255 instead of 130 (128 + 2 for
SIGINT).

Could you elaborate on the actual issue here?

It exits, that's the goal.

PID 1 processes must install their own signal handlers, and we do so:

https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Unix.hs#L23-L31

If we didn't do that, PostgREST wouldn't exit at all.

However, this suggests that PostgREST itself may have
issues with handling signals or properly shutting down its threads.

How do threads need to be shutdown properly? We are not spawning subprocesses, so I fail to see what we need to do here.

I suppose the problem occurs in a much simpler setup, but to be sure you can
repeat it

Do:

docker compose run --rm postgrest/postgrest:latest

hit CTRL+C. Observe - the exit code is 255.

Note: When doing the same by running postgrest outside the docker container, I get an exit code of 130.

So yeah... apparently we still do something different. Not sure what, though. My assumption is that when running as pid > 1, the default signal handlers kick in and return 130 as the exit code, while the way we implemented our custom handlers for the pid 1 case ends up with 255 as the exit code.

[m-ickiewicz]

Could you elaborate on the actual issue here?

You can close gracefully (exit with code 130 or 0) or not. When a program exits with an error code (i.e., an exit code different from the expected one), supervisor scripts (e.g., health checks, service management scripts) may trigger rescue actions. Fortunately, this is not an issue in my case—I can live with this error.

Closing with an error means that the shutdown process was interrupted and may not have completed correctly (e.g., some resources might not have been freed). I am unsure whether PostgREST manages to clean up everything it needs to during exit—maybe it does, maybe it doesn't. I also don't know the consequences of this issue for PostgREST itself.

It is better to close threads properly rather than risk being caught by some random issues. I reported this issue because I couldn't find any information about it, and I want to make you aware that it exists. However, I don't know how important this is to you.

[steve-chavez]

I recall there was a problem with CTRL + C on docker not killing PostgREST, this is the same right? (reading https://gitlab.syncad.com/hive/hivemind/-/issues/277, looks like it is)

Fortunately, this is not an issue in my case—I can live with this error.
However, I don't know how important this is to you.

We'll track it as "minor" bug for now, thanks for reporting.