acme.sh persists the --install-cert paths and replays them on every cron renewal, where cwd is $HOME rather than the repo. The relative default resolved to ~/certs/... so renewals wrote nothing: acme.sh kept renewing cleanly while nginx served the stale copy until it expired. Apex, auth. and pack. served expired certs on 2026-09-06. Absolutise CERT_DIR in issue-letsencrypt.sh and document the failure mode, the recovery command, and off-host verification in plan/15-letsencrypt.md. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018UJzZrBaZUoU6qHMLpLz64
4.5 KiB
TLS — Let's Encrypt (OVH DNS-01) + host nginx ingress
Production serves real, publicly-trusted certs via the host's own nginx.
nginx terminates TLS and reverse-proxies every vhost to the caddy container
(published localhost-only on ${CADDY_HTTP_PORT}) by Host header. caddy is the
internal router; nginx is the public TLS terminator. This is the only ingress
path — there are no compose overrides.
Certs: one per name, via OVH DNS-01
tooling/issue-letsencrypt.sh issues a separate cert for the apex and each
subdomain (${BASE_DOMAIN}, auth.…, plus the rest of
LE_SUBDOMAINS) using acme.sh + the OVH DNS-01 challenge — no inbound ports
needed, so it works regardless of public reachability.
One-time setup
- Install acme.sh:
curl https://get.acme.sh | sh -s email=you@example.com - OVH API credentials →
.env(see.env.exampleOVH_*,LE_EMAIL,LE_SUBDOMAINS). Create a token at https://eu.api.ovh.com/createToken/ withGET/POST/PUT/DELETEon/domain/zone/*, or leaveOVH_CKblank and run the script once — acme.sh prints an authorization URL; visit it, then paste the consumer key intoOVH_CKand re-run.
Issue
tooling/issue-letsencrypt.sh # real certs
LE_STAGING=1 tooling/issue-letsencrypt.sh # dry run against the LE staging CA
Certs install to certs/<name>/{cert.pem,key.pem} (gitignored). acme.sh adds a
cron for renewal; --reloadcmd reloads nginx automatically on renew. Override
the reload with RELOAD_CMD=... if nginx isn't systemd-managed.
Gotcha — install paths must be ABSOLUTE (silent expiry, 2026-09-06)
--install-cert paths are persisted in acme.sh's domain conf
(~/.acme.sh/<name>_ecc/<name>.conf → Le_RealKeyPath /
Le_RealFullChainPath) and replayed on every cron renewal. The cron runs with
cwd=$HOME, not the repo — so a relative path (certs/<name>/key.pem)
resolves to ~/certs/... and the install writes nothing.
Failure mode is silent and delayed: acme.sh --list shows every cert freshly
renewed with a future Renew date, ~/.acme.sh/<name>_ecc/fullchain.cer is
valid — while certs/<name>/cert.pem is frozen at the last manual run and
nginx serves that until it expires. There is no error anywhere. Cost: apex,
auth. and pack. served expired certs on 2026-09-06.
Fix / recovery — re-run --install-cert with absolute paths, which both copies
the current cert and rewrites the persisted paths:
APP=/home/ubuntu/mc/ulicraft-server-v1
for d in $(ls $APP/certs); do
~/.acme.sh/acme.sh --install-cert -d "$d" --ecc \
--key-file "$APP/certs/$d/key.pem" \
--fullchain-file "$APP/certs/$d/cert.pem" \
--reloadcmd "sudo systemctl reload nginx || true"
done
issue-letsencrypt.sh now absolutises CERT_DIR so a fresh issue can't
reintroduce it.
Verify from off-host, not from the files — the installed file and what nginx serves are the same thing here, but the acme.sh copy is not:
for h in ulicraft.net auth.ulicraft.net avatar.ulicraft.net status.ulicraft.net \
distribution.ulicraft.net files.ulicraft.net upload.ulicraft.net \
monitor.ulicraft.net generators.ulicraft.net; do
printf '%-30s ' "$h"
echo | openssl s_client -servername $h -connect $h:443 2>/dev/null \
| openssl x509 -noout -enddate
done
nginx
- Bring the stack up (caddy publishes on
127.0.0.1:${CADDY_HTTP_PORT}):docker compose up -d --build - Render + install the vhost (
nginx/ulicraft-caddy.conf.tmpl):This substitutestooling/render-nginx.sh --install$BASE_DOMAIN $APP_DIR $CADDY_HTTP_PORT, writes tosites-available, symlinkssites-enabled, runsnginx -t, and reloads nginx. nginx then terminates TLS per vhost (certs under$APP_DIR/certs) and reverse-proxies each Host to caddy on127.0.0.1:${CADDY_HTTP_PORT}. Ensurewww-datacan read$APP_DIR/certs.
DNS
Point the public DNS for ${BASE_DOMAIN} and every subdomain
(auth. avatar. status. distribution. www.) at the host running
nginx. DNS is configured outside this repo. The Minecraft container reaches
the stack's own names via caddy's mcnet aliases, not public DNS.
Why DNS-01 (not HTTP-01)
DNS-01 only needs the OVH API, so it issues regardless of public reachability —
and ulicraft.net is a real OVH-managed domain, which LE requires (it never
issues for .lan/.local). HTTP-01/TLS-ALPN-01 would need public inbound on
80/443. The JVM trusts the resulting Let's Encrypt certs with no CA import.