Document the gotcha hit fixing distribution CORS: caddy/conf.d/*.caddy are single-file bind mounts pinned to the host inode at container creation. git pull rewrites tracked files via atomic rename (new inode), so a running caddy keeps the old config; `restart`/`caddy reload` reuse the stale inode. Apply conf changes with `docker compose up -d --force-recreate caddy` (or the --hard down/up). Added to plan/14-deploy.md and the deploy skill guardrails. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
153 lines
7.0 KiB
Markdown
153 lines
7.0 KiB
Markdown
# Deployment — redeploy to `cochi`
|
||
|
||
Production host for the Ulicraft stack.
|
||
|
||
| | |
|
||
|---|---|
|
||
| Host | `cochi` (SSH alias) |
|
||
| Path | `/home/ubuntu/mc/ulicraft-server-v1` |
|
||
| Branch | `main` |
|
||
| Compose | single `docker-compose.yml` (no overrides) |
|
||
| Auth | Drasl (password login) |
|
||
| Ingress | host nginx (Let's Encrypt TLS) → caddy → services (see 15-letsencrypt.md) |
|
||
| Restart | **hard** (`compose down` → `compose up --build`) — brief full downtime |
|
||
|
||
## Routine redeploy
|
||
|
||
`update-all.sh` (repo root) is the **single source of truth** for the post-pull
|
||
build + restart steps. Run it AFTER a `git pull` (it never pulls or pushes).
|
||
Strict halt on any error — never a partial deploy.
|
||
|
||
Two modes:
|
||
|
||
- `./update-all.sh` — **rolling**: `docker compose up -d --build`, then restart
|
||
ONLY the services whose mounted inputs changed since a pre-run snapshot
|
||
(drasl/nmsr on a config re-render, minecraft on a mod change). Minimal
|
||
downtime. Does **not** detect caddy static-conf (`caddy/conf.d/*.caddy`)
|
||
changes — apply those with a **recreate**, not a reload/restart (see gotcha
|
||
below): `docker compose up -d --force-recreate caddy` (no world downtime), or
|
||
`--hard` for a full redeploy.
|
||
|
||
> **Caddy conf bind mounts are inode-pinned — recreate, don't reload.** Each
|
||
> `caddy/conf.d/*.caddy` is a *single-file* bind mount, which Docker pins to
|
||
> the host file's inode at container creation. `git pull` rewrites a tracked
|
||
> file via atomic rename = a NEW inode, so the running container keeps the OLD
|
||
> config. `docker compose restart caddy` and `caddy reload` reuse the same
|
||
> container → same stale inode → no effect. Only re-creating the container
|
||
> (`up -d --force-recreate caddy`, or the `--hard` down/up) re-resolves the
|
||
> mount. Verify: `docker compose exec caddy cat /etc/caddy/conf.d/<file>`.
|
||
- `./update-all.sh --hard` — `docker compose down --remove-orphans && up -d
|
||
--build`. Full restart (Minecraft world downtime); picks up everything.
|
||
|
||
One-shot full redeploy from your workstation (the `deploy` skill does exactly
|
||
this):
|
||
|
||
```bash
|
||
ssh cochi 'set -e
|
||
cd /home/ubuntu/mc/ulicraft-server-v1
|
||
git pull --ff-only
|
||
tooling/fetch-authlib.sh # ensure authlib jar (rarely changes; NOT in update-all)
|
||
./update-all.sh --hard'
|
||
```
|
||
|
||
Everyday landing/config tweak (no world downtime):
|
||
|
||
```bash
|
||
ssh cochi 'cd /home/ubuntu/mc/ulicraft-server-v1 && git pull --ff-only && ./update-all.sh'
|
||
```
|
||
|
||
Steps `update-all.sh` runs, in order: `render-config.sh` (drasl/nmsr from `.env`;
|
||
halts on a bad `REGISTRATION_MODE`) → `sync-server-mods.sh` (mirror the
|
||
`both`/`server` subset from `mods-sides.json` out of `$DISTRIBUTION_WEB_ROOT`
|
||
into `./server-mods`) → `build-modlist.py` (regen landing `mods.json` + logos;
|
||
**run under `python>=3.11`** — cochi's default `python3` is 3.10, but
|
||
`python3.11` is installed and the script auto-selects it) → landing `pnpm build`
|
||
(sources nvm + `.env`, never `npm`) → docker compose. `$DISTRIBUTION_WEB_ROOT`
|
||
must resolve on the host (it also backs the caddy `distribution.` mount).
|
||
|
||
### Landing site + mod list (generated, not in git)
|
||
|
||
Both the static site `www/` AND the landing `mods.json` are gitignored — a `git
|
||
pull` never updates them. `update-all.sh` regenerates both every run
|
||
(`mods.json` via `build-modlist.py`, `www/` via `pnpm build`). A flag-only
|
||
`REGISTRATION_MODE` change just needs a rolling `./update-all.sh`: it rebuilds
|
||
`www/` (bakes the invite-field on/off) and restarts `drasl` only if
|
||
`render-config.sh` changed its config (Drasl's `RequireInvite`/CORS/RateLimit) —
|
||
no world downtime.
|
||
|
||
## Known host-local divergence on `cochi` (uncommitted)
|
||
|
||
`cochi` carries deliberate local edits that are **not in the repo**, so
|
||
`git pull --ff-only` **aborts** ("local changes would be overwritten") whenever
|
||
an incoming commit also touches `docker-compose.yml`. As of 2026-06-09 the host
|
||
has, uncommitted:
|
||
|
||
- `docker-compose.yml` — internal URLs forced `http`→`https`
|
||
(`JVM_OPTS` authlib-injector), for the in-container HTTPS path. (The old
|
||
`PACKWIZ_URL` divergence is moot — packwiz was removed; see `04-mods.md`.)
|
||
- `dnsmasq/` (new), `caddy/caddy-root-ca.crt`, `landing/pnpm-workspace.yaml` —
|
||
internal DNS + private CA so containers resolve/trust `https://auth.` etc.
|
||
- `landing/package.json` modified. (The former `pack/` divergence is resolved —
|
||
`pack/` and `custom/` were deleted with the packwiz removal.)
|
||
|
||
**Do not steamroll it.** Before a deploy that edits `docker-compose.yml`, either:
|
||
1. **Converge** — commit the host edits to `main` + push, then a clean ff-pull
|
||
deploys (preferred — ends the divergence). The https/dnsmasq/CA setup belongs
|
||
in the repo.
|
||
2. **Stash–pull–pop** — `git stash` → `git pull --ff-only` → `git stash pop`.
|
||
Non-destructive; safe only when the incoming and host edits are in different
|
||
regions of the file. Stop on a pop conflict.
|
||
|
||
Never `git reset --hard` / discard on the host without confirming first.
|
||
|
||
## What survives a hard restart
|
||
|
||
Named volumes persist — world and auth data are safe:
|
||
- `mc_data` — world + installed NeoForge/mods
|
||
- `drasl_state` — Drasl accounts/skins
|
||
- `kuma_data` — Uptime Kuma config/history
|
||
- `backups/` — mc-backup snapshots
|
||
|
||
`down` removes **containers**, not volumes. Players are disconnected during the
|
||
restart (a few minutes); they reconnect once `minecraft` is healthy again.
|
||
|
||
## Prerequisites (one-time, on `cochi`)
|
||
|
||
- SSH access as the `cochi` alias.
|
||
- `.env` present and complete: `BASE_DOMAIN`, `RCON_PASSWORD`, `CADDY_HTTP_PORT`,
|
||
`CADDY_HTTP_BIND`, `DISTRIBUTION_WEB_ROOT`, `REGISTRATION_MODE` (`invite`|`open`,
|
||
defaults `invite`), and the Let's Encrypt block (`render-config.sh` halts on an
|
||
unset `BASE_DOMAIN` or an invalid `REGISTRATION_MODE`).
|
||
- Docker + compose plugin, `envsubst`, `jq` (used by `sync-server-mods.sh`;
|
||
falls back to `python3` if absent), `git`.
|
||
- Host nginx + Let's Encrypt certs installed once — see `15-letsencrypt.md`.
|
||
- **First deploy only** — online prep (build landing, fetch launcher, fetch
|
||
authlib into `runtime/`). Not needed for routine redeploys.
|
||
|
||
## Verify
|
||
|
||
```bash
|
||
docker compose ps # caddy, drasl, minecraft, mc-backup, nmsr, mc-status, uptime-kuma up
|
||
# self-hosted live status feed (plan/15-mc-status.md):
|
||
curl -fsS "https://$(grep '^BASE_DOMAIN=' .env | cut -d= -f2)/api/mcstatus/v2/status/java/$(grep '^BASE_DOMAIN=' .env | cut -d= -f2)" | jq .online
|
||
docker compose logs -f minecraft # watch for "Done"
|
||
curl -fsS "https://auth.$(grep '^BASE_DOMAIN=' .env | cut -d= -f2)/" >/dev/null && echo "auth ok"
|
||
```
|
||
|
||
## Rollback
|
||
|
||
```bash
|
||
git log --oneline -5 # find the last-good commit
|
||
git checkout <sha> # or: git reset --hard <sha>
|
||
docker compose down --remove-orphans
|
||
docker compose up -d --build
|
||
```
|
||
|
||
## Notes
|
||
|
||
- Production needs internet on `cochi` at restart: the server installs/re-syncs
|
||
NeoForge + mods over the network.
|
||
- All ingress goes through host nginx → caddy. After a redeploy, nginx config is
|
||
unchanged; only re-run `tooling/render-nginx.sh --install` if `BASE_DOMAIN` /
|
||
`CADDY_HTTP_PORT` changed (see `15-letsencrypt.md`).
|