Initialise embedded PostgreSQL where a platform volume lets it

CHANGELOG.md

@@ -8,6 +8,34 @@ Newest first. `Unreleased` is what is on `main` and not yet tagged.
 
 ## Unreleased
 
+### Embedded PostgreSQL initialises on a platform volume, and says so when it cannot
+
+`EMBEDDED_POSTGRES=on` could not create its cluster on a platform whose persistent volume is an ext4
+mount — Railway, and by the same mechanism most others — and it failed differently depending on where
+the volume was mounted. Neither of the two paths this repo suggested worked, and the two suggestions
+disagreed with each other: `docs/deployment.md` said `/var/lib/postgresql/data`, the `Dockerfile`
+comment said `/var/lib/postgresql`.
+
+Mounted at the parent, the mount arrives owned by root, `data` is not in it, and the image's
+build-time `chown` is hidden underneath — so `initdb`, which has already dropped to the `postgres`
+user, cannot create the directory. `postgres-init` now creates and chowns it first, as root, which is
+the only step in a position to. This also fixes the plain
+`docker run -v openbot-data:/var/lib/postgresql` case, which relied entirely on that hidden chown.
+
+Mounted directly on the data directory, the mount arrives holding a `lost+found`, and `initdb` will
+not initialise into a directory with anything in it. **The documented mount is now the parent,
+`/var/lib/postgresql`**, which leaves `data` an ordinary subdirectory — what PostgreSQL's own hint
+asks for, and what the `Dockerfile` already said. A volume already mounted at
+`/var/lib/postgresql/data` and working — a Docker named volume, which arrives empty rather than with a
+`lost+found` — keeps working and needs no change.
+
+**The failure said nothing useful.** `api` waits on `postgres` and `migrate`, so neither started, the
+container came up anyway, the platform reported the deploy a success, and the public URL served a
+persistent 502 with the real reason visible only in the container log. A data directory that holds no
+cluster and is not empty is now refused with a sentence naming the mount to use instead.
+
+Reported by [@jerelvelarde](https://github.com/CopilotKit/OpenBot/issues/269) with the container logs
+for both mount paths, which is what made the two failure modes separable.
 ### A sign-in a site opens in a new window is shown, and can be clicked
 
 A Bot's browser was bound to the page it launched with, and to nothing the site opened afterwards.

docker/s6/scripts/postgres-init.sh

@@ -9,6 +9,30 @@ set -eu
 
 DATA=/var/lib/postgresql/data
 if [ ! -s "$DATA/PG_VERSION" ]; then
+  # Created and owned here, as root, because this is the only step in a position to do it.
+  #
+  # The image creates and chowns this at build time, and a volume mounted over /var/lib/postgresql at
+  # run time hides that completely: the mount arrives owned by root, `data` is not in it, and nothing
+  # else in the image puts either back. There is no `fix-attrs.d` under docker/s6, so the built-in
+  # s6-overlay service of that name has nothing to act on. `postgres-init` is a oneshot whose `up`
+  # runs as root, so it can, and `initdb` a line below cannot — it has already dropped to `postgres`.
+  mkdir -p "$DATA"
+  chown postgres:postgres "$DATA"
+
+  # A volume mounted directly AT the data directory, rather than at its parent, arrives holding
+  # `lost+found` on any platform whose volume is an ext4 mount — which is most of them. `initdb`
+  # refuses a directory with anything in it, and its own hint says to use a subdirectory instead.
+  #
+  # Said here, naming this image's answer, rather than left to that hint. The failure is otherwise a
+  # generic message about mount points in a container log, while `api` never starts because it depends
+  # on `postgres` and `migrate`, the platform reports the deploy a success, and the public URL serves
+  # a 502 with nothing on it to explain why.
+  if [ -n "$(ls -A "$DATA" 2>/dev/null)" ]; then
+    echo "postgres-init: $DATA holds no cluster and is not empty, so initdb cannot use it." >&2
+    echo "postgres-init: mount the volume at /var/lib/postgresql rather than at $DATA. A volume mounted directly on the data directory arrives with a lost+found in it, and PostgreSQL will not initialise into that." >&2
+    exit 1
+  fi
+
   s6-setuidgid postgres /usr/lib/postgresql/16/bin/initdb -D "$DATA" -A trust -U openbot >/dev/null
   s6-setuidgid postgres /usr/lib/postgresql/16/bin/pg_ctl -D "$DATA" -o "-c listen_addresses=127.0.0.1" -w start >/dev/null
   s6-setuidgid postgres /usr/lib/postgresql/16/bin/createdb -U openbot openbot

docs/deployment.md

@@ -11,7 +11,7 @@ docker run -p 3001:3001 --env-file .env openbot
 
 # Or one inside the container. Nothing else to provision.
 docker run -p 3001:3001 --env-file .env \
-  -e EMBEDDED_POSTGRES=on -v openbot-data:/var/lib/postgresql/data openbot
+  -e EMBEDDED_POSTGRES=on -v openbot-data:/var/lib/postgresql openbot
 ```
 
 ## What is in the image, and what is not
@@ -24,8 +24,16 @@ process beside it.
 the database and the `vector` extension the first time, and runs the migrations on every start. It
 listens on loopback only and is never published, so there is no password to manage.
 
-Give it a volume at `/var/lib/postgresql/data`. Without one, a redeploy takes the audit trail with
-it, and the audit trail is the product. Platforms that offer no persistent volume are the ones to
+Give it a volume at `/var/lib/postgresql` — the parent, not the data directory itself. Without one,
+a redeploy takes the audit trail with it, and the audit trail is the product.
+
+**Mount the parent, not `/var/lib/postgresql/data`.** On any platform whose volume is an ext4 mount,
+and that is most of them, the mount arrives holding a `lost+found` directory. `initdb` will not
+initialise into a directory that has anything in it, so mounting it directly on the data directory
+leaves the cluster uncreated — and because `api` waits on `postgres` and `migrate`, the container
+starts, the platform reports the deploy a success, and the URL serves a 502. Mounting the parent
+leaves `data` as an ordinary subdirectory, which is what PostgreSQL asks for. A Docker named volume
+works either way; a platform volume does not. Platforms that offer no persistent volume are the ones to
 point at a managed database instead: set `DATABASE_URL` and leave `EMBEDDED_POSTGRES` off. The
 `vector` extension must be enabled there; RDS, Cloud SQL and Azure Database all support it, none
 enable it for you.