docs: fold the monitor service into the stack-level docs
plan/23-monitor.md already covered the design; this makes the monitor discoverable from the places someone actually starts reading. - README: stack table, vhost list, DNS names, layout, and a setup section covering the bcrypt credential, the no-published-port model, and the --hard-on-first-deploy requirement. - plan/00-overview.md: drop (PLANNED), add to the always-on service list, and note that it starts fine with Minecraft down (missing log and refused RCON are reported states, not fatal). - plan/19-routes.md: vhost row, the four GET routes with their contracts, and the internal mcnet edges (mc-status for rosters so it needs no Drasl creds, RCON for TPS, Kuma probing healthz around caddy). - caddy/Caddyfile: the snippet index had drifted — it listed five of nine vhosts. Added the four missing ones. Two gotchas promoted to CLAUDE.md because they cost real time and will recur: - Compose interpolates $-sequences in .env VALUES, so a bare bcrypt hash is silently corrupted before reaching the container, and the same bug bites again when writing one to a remote .env over ssh. Also records that caddy and filestash want opposite hash formats. - The monitor container runs as root deliberately: /data is drwxr-x--- owned by uid 1000, so a USER directive would make the tailer silently read nothing while still returning 200. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
bc99427ef9
commit
bac2372c95
5 files changed
+119
-4
No files matched your search
@@ -37,6 +37,8 @@ Internet ── host nginx (TLS, LE certs, HSTS) ──► caddy (127.0.0.1:8880
|
|||||||
upload.${BASE_DOMAIN} ─► filestash (player file share, ONE shared login, writable)
|
upload.${BASE_DOMAIN} ─► filestash (player file share, ONE shared login, writable)
|
||||||
files.${BASE_DOMAIN} ─► caddy (read-only, no-auth browse of the same share dir;
|
files.${BASE_DOMAIN} ─► caddy (read-only, no-auth browse of the same share dir;
|
||||||
per-host on/off via FILES_BROWSE — off on cochi)
|
per-host on/off via FILES_BROWSE — off on cochi)
|
||||||
|
monitor.${BASE_DOMAIN}─► monitor (ADMIN ONLY: latest.log stream + dashboard tiles,
|
||||||
|
caddy basic_auth; per-host on/off MONITOR_ENABLED)
|
||||||
distribution.${BASE} ─► static site from ANOTHER repo (DISTRIBUTION_WEB_ROOT)
|
distribution.${BASE} ─► static site from ANOTHER repo (DISTRIBUTION_WEB_ROOT)
|
||||||
|
|
||||||
minecraft :25565 (+ 24454/udp Simple Voice Chat)
|
minecraft :25565 (+ 24454/udp Simple Voice Chat)
|
||||||
@@ -153,6 +155,34 @@ works. After any filestash config change, check the browser-facing values
|
|||||||
the broken redirect announced itself there (`msg=Redirecting to http://https://`)
|
the broken redirect announced itself there (`msg=Redirecting to http://https://`)
|
||||||
while every other check passed.
|
while every other check passed.
|
||||||
|
|
||||||
|
### Compose eats `$` in `.env` values — quote every hash
|
||||||
|
`docker compose` interpolates `$`-sequences inside `.env` **values**, so a bare
|
||||||
|
bcrypt hash is silently corrupted before it ever reaches a container:
|
||||||
|
|
||||||
|
```
|
||||||
|
MONITOR_PASSWORD_HASH=$2a$14$abcdefgHIJKLmnop.qrstuv # WRONG
|
||||||
|
→ container sees $2a$14.qrstuv # $abcdefg… eaten as an unset var
|
||||||
|
MONITOR_PASSWORD_HASH='$2a$14$abcdefgHIJKLmnop.qrstuv' # single quotes fix it
|
||||||
|
```
|
||||||
|
|
||||||
|
Verified, not folklore. Every login fails with no hint as to why — the hash
|
||||||
|
looks fine in `.env` and `docker compose config` re-escapes it to `$$…`, which
|
||||||
|
looks wrong but is correct. **The same bug bites over SSH**: writing a hash to a
|
||||||
|
remote `.env` through `ssh host "...$HASH..."` lets the remote shell expand it.
|
||||||
|
Transport it base64-encoded and verify with `md5sum` on both ends.
|
||||||
|
|
||||||
|
Note the two services want *opposite* formats: caddy `basic_auth` needs
|
||||||
|
**bcrypt** (`caddy hash-password`), filestash needs **`$6$`** (`openssl passwd
|
||||||
|
-6`) and silently rejects bcrypt. Never copy a hash between them.
|
||||||
|
|
||||||
|
### Monitor runs as root on purpose
|
||||||
|
`/data` is `drwxr-x---` owned by uid 1000 (itzg). A non-root container user
|
||||||
|
cannot traverse it, so the log tailer would read nothing — **silently**, while
|
||||||
|
still answering HTTP 200. Adding a `USER` directive to `docker/monitor/` as
|
||||||
|
"hardening" breaks it. What actually bounds the risk there: every mount is `:ro`,
|
||||||
|
no port is published, and the one credential it holds (RCON) can only send
|
||||||
|
commands from a compile-time allowlist.
|
||||||
|
|
||||||
### Memory sizing
|
### Memory sizing
|
||||||
~50–100 mods, 8 players, 1.21.1: `INIT_MEMORY: 4G`, `MAX_MEMORY: 10G`,
|
~50–100 mods, 8 players, 1.21.1: `INIT_MEMORY: 4G`, `MAX_MEMORY: 10G`,
|
||||||
`USE_AIKAR_FLAGS: TRUE`. **The pack is now 164 jars / 133 server-side** (Create
|
`USE_AIKAR_FLAGS: TRUE`. **The pack is now 164 jars / 133 server-side** (Create
|
||||||
@@ -219,6 +249,10 @@ Full design settled across WS1–WS6; all tasks unchecked, ready to execute.
|
|||||||
htpasswd login, writable, rendered `:ro` config. See `plan/20-files.md`.
|
htpasswd login, writable, rendered `:ro` config. See `plan/20-files.md`.
|
||||||
Follow-ups: add the Uptime Kuma HTTP monitor for `files.`; the share is
|
Follow-ups: add the Uptime Kuma HTTP monitor for `files.`; the share is
|
||||||
**outside mc-backup** and unquota'd (accepted — eyeballed).
|
**outside mc-backup** and unquota'd (accepted — eyeballed).
|
||||||
|
- [x] **Monitor** (`monitor.`, 2026-08-01) — admin-only `latest.log` stream +
|
||||||
|
dashboard tiles (status, TPS via allowlisted RCON, players, disk, backups,
|
||||||
|
joins/leaves). Live on cochi, all three phases done. See `plan/23-monitor.md`.
|
||||||
|
Follow-up: add the Uptime Kuma monitor for `http://monitor:8080/healthz`.
|
||||||
- [ ] **Verify NeoForge version** against current stable before first deploy.
|
- [ ] **Verify NeoForge version** against current stable before first deploy.
|
||||||
- [x] **Bootstrap Drasl admin** account (`admin`, 2026-06-10). Key is
|
- [x] **Bootstrap Drasl admin** account (`admin`, 2026-06-10). Key is
|
||||||
`DefaultAdmins` (not `DefaultAdminUsernames`); first admin needs an invite-flip.
|
`DefaultAdmins` (not `DefaultAdminUsernames`); first admin needs an invite-flip.
|
||||||
|
|||||||
@@ -19,6 +19,7 @@ One compose file, one `docker compose up -d`:
|
|||||||
| `uptime-kuma` | louislam/uptime-kuma | status page at `status.${BASE_DOMAIN}` |
|
| `uptime-kuma` | louislam/uptime-kuma | status page at `status.${BASE_DOMAIN}` |
|
||||||
| `filestash` | machines/filestash (digest-pinned) | player file share (upload) at `upload.${BASE_DOMAIN}` |
|
| `filestash` | machines/filestash (digest-pinned) | player file share (upload) at `upload.${BASE_DOMAIN}` |
|
||||||
| `mc-backup` | itzg/mc-backup | world backups every 6h via RCON |
|
| `mc-backup` | itzg/mc-backup | world backups every 6h via RCON |
|
||||||
|
| `monitor` | built from source | admin-only log stream + dashboard at `monitor.${BASE_DOMAIN}` |
|
||||||
|
|
||||||
Vhosts (all proxied through the host nginx → caddy):
|
Vhosts (all proxied through the host nginx → caddy):
|
||||||
|
|
||||||
@@ -27,6 +28,7 @@ Vhosts (all proxied through the host nginx → caddy):
|
|||||||
- `avatar.` — NMSR
|
- `avatar.` — NMSR
|
||||||
- `status.` — Uptime Kuma
|
- `status.` — Uptime Kuma
|
||||||
- `files.` — Filestash player file share (one shared login, writable)
|
- `files.` — Filestash player file share (one shared login, writable)
|
||||||
|
- `monitor.` — **admin only**, log stream + dashboard (caddy basic_auth)
|
||||||
- `distribution.` — static site from another repo (`DISTRIBUTION_WEB_ROOT`)
|
- `distribution.` — static site from another repo (`DISTRIBUTION_WEB_ROOT`)
|
||||||
|
|
||||||
Config lives in `.env`: `BASE_DOMAIN`, `RCON_PASSWORD`, `CADDY_HTTP_PORT` /
|
Config lives in `.env`: `BASE_DOMAIN`, `RCON_PASSWORD`, `CADDY_HTTP_PORT` /
|
||||||
@@ -35,7 +37,8 @@ Config lives in `.env`: `BASE_DOMAIN`, `RCON_PASSWORD`, `CADDY_HTTP_PORT` /
|
|||||||
## DNS
|
## DNS
|
||||||
|
|
||||||
Handled **outside this repo**. Point `${BASE_DOMAIN}` and every subdomain
|
Handled **outside this repo**. Point `${BASE_DOMAIN}` and every subdomain
|
||||||
(`auth. avatar. status. files. distribution. www.`) at the host running nginx.
|
(`auth. avatar. status. files. monitor. distribution. www.`) at the host running
|
||||||
|
nginx.
|
||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
@@ -158,6 +161,43 @@ curl -H "Host: ulicraft.net" \
|
|||||||
|
|
||||||
Wired in `landing/src/data/site.ts` (`statusApi`) and `caddy/conf.d/10-static.caddy`.
|
Wired in `landing/src/data/site.ts` (`statusApi`) and `caddy/conf.d/10-static.caddy`.
|
||||||
|
|
||||||
|
## Monitor (admin log stream + dashboard)
|
||||||
|
|
||||||
|
`monitor.${BASE_DOMAIN}` — a small Go service (`docker/monitor/`) that tails the
|
||||||
|
Minecraft server's `latest.log` and shows it live with autoscroll, a filter and
|
||||||
|
ERROR/WARN/INFO chips, above a row of tiles: server status, TPS, players online
|
||||||
|
(with NMSR heads), registered accounts, disk free, backup freshness and recent
|
||||||
|
joins/leaves.
|
||||||
|
|
||||||
|
**Admin only, and read-only.** The log carries player IPs, chat and admin
|
||||||
|
commands verbatim and nothing sanitizes it, so the whole vhost sits behind caddy
|
||||||
|
`basic_auth`. There are no controls in the UI — every route is a `GET`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# generate the credential (BCRYPT — the opposite of FILES_PASSWORD_HASH, which needs $6$)
|
||||||
|
docker run --rm caddy:alpine caddy hash-password --plaintext 'your-password'
|
||||||
|
```
|
||||||
|
|
||||||
|
Then in `.env` — **single-quote the hash**, Compose mangles bare `$`-sequences:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
MONITOR_ENABLED=on # per-host on|off, like FILES_BROWSE
|
||||||
|
MONITOR_USER=admin
|
||||||
|
MONITOR_PASSWORD_HASH='$2a$14$...'
|
||||||
|
```
|
||||||
|
|
||||||
|
The container publishes **no port**: caddy is the only way in, so there is no
|
||||||
|
host listener to firewall. It holds one credential (`RCON_PASSWORD`, for the TPS
|
||||||
|
tile) and can only send commands from a compile-time allowlist that no HTTP
|
||||||
|
input can reach.
|
||||||
|
|
||||||
|
Adding it needs a cert (`monitor` in `LE_SUBDOMAINS` →
|
||||||
|
`tooling/issue-letsencrypt.sh`) and an nginx vhost (`tooling/render-nginx.sh
|
||||||
|
--install`). Because it adds a caddy conf file, the **first** deploy needs
|
||||||
|
`./update-all.sh --hard` (world downtime); later changes are rolling.
|
||||||
|
|
||||||
|
Full design, gotchas and verification: **`plan/23-monitor.md`**.
|
||||||
|
|
||||||
## Player file share (Filestash)
|
## Player file share (Filestash)
|
||||||
|
|
||||||
`upload.${BASE_DOMAIN}` is a web file share for player media — screenshots,
|
`upload.${BASE_DOMAIN}` is a web file share for player media — screenshots,
|
||||||
@@ -224,6 +264,8 @@ caddy/Caddyfile + conf.d/*.caddy # ingress vhost snippets
|
|||||||
drasl/config/config.toml.tmpl # auth config (rendered)
|
drasl/config/config.toml.tmpl # auth config (rendered)
|
||||||
nmsr/config.toml.tmpl # avatar renderer config, Drasl-backed (rendered)
|
nmsr/config.toml.tmpl # avatar renderer config, Drasl-backed (rendered)
|
||||||
docker/nmsr/ # built-from-source NMSR image
|
docker/nmsr/ # built-from-source NMSR image
|
||||||
|
docker/mc-status/ # built-from-source SLP→JSON pinger
|
||||||
|
docker/monitor/ # built-from-source admin log stream + dashboard
|
||||||
landing/ # Astro site → www/
|
landing/ # Astro site → www/
|
||||||
launcher/ # FjordLauncher assets (gitignored)
|
launcher/ # FjordLauncher assets (gitignored)
|
||||||
server-mods/ # filtered server mod jars (synced, gitignored)
|
server-mods/ # filtered server mod jars (synced, gitignored)
|
||||||
|
|||||||
@@ -4,6 +4,10 @@
|
|||||||
# conf.d/30-distribution.caddy static site from another repo
|
# conf.d/30-distribution.caddy static site from another repo
|
||||||
# conf.d/40-avatar.caddy nmsr skin/avatar renderer
|
# conf.d/40-avatar.caddy nmsr skin/avatar renderer
|
||||||
# conf.d/50-status.caddy uptime-kuma status page
|
# conf.d/50-status.caddy uptime-kuma status page
|
||||||
|
# conf.d/60-upload.caddy filestash player file share
|
||||||
|
# conf.d/62-files.caddy read-only browse of the share (FILES_BROWSE)
|
||||||
|
# conf.d/70-generators.caddy offline misode data-pack generators
|
||||||
|
# conf.d/80-monitor.caddy admin log stream (basic_auth, MONITOR_ENABLED)
|
||||||
# All snippets are plain http:// on :80 — the host's nginx terminates TLS in
|
# All snippets are plain http:// on :80 — the host's nginx terminates TLS in
|
||||||
# front (nginx/ulicraft-caddy.conf.tmpl). disable_redirects keeps caddy from
|
# front (nginx/ulicraft-caddy.conf.tmpl). disable_redirects keeps caddy from
|
||||||
# upgrading to its own https.
|
# upgrading to its own https.
|
||||||
|
|||||||
+6
-3
@@ -152,10 +152,13 @@ mc-status, uptime-kuma and filestash.
|
|||||||
- `18-landing-rework.md` — join flow / rosters / account / mod list / footer (PLANNED)
|
- `18-landing-rework.md` — join flow / rosters / account / mod list / footer (PLANNED)
|
||||||
- `19-routes.md` — HTTP route reference (nmsr / landing / auth / files + internal)
|
- `19-routes.md` — HTTP route reference (nmsr / landing / auth / files + internal)
|
||||||
- `20-files.md` — Filestash player file share (`files.`, one shared login)
|
- `20-files.md` — Filestash player file share (`files.`, one shared login)
|
||||||
- `23-monitor.md` — admin log stream + dashboard (`monitor.`, basic auth) (PLANNED)
|
- `23-monitor.md` — admin log stream + dashboard (`monitor.`, basic auth)
|
||||||
|
|
||||||
## Boot/dependency order
|
## Boot/dependency order
|
||||||
|
|
||||||
`caddy` (aliases) → `drasl` → `minecraft` (depends on drasl) → `mc-backup`.
|
`caddy` (aliases) → `drasl` → `minecraft` (depends on drasl) → `mc-backup`.
|
||||||
`nmsr`, `mc-status`, `uptime-kuma` and `filestash` are always-on and join `mcnet`
|
`nmsr`, `mc-status`, `uptime-kuma`, `filestash` and `monitor` are always-on and
|
||||||
(no ordering constraints — none of them are on the auth/boot path).
|
join `mcnet` (no ordering constraints — none of them are on the auth/boot path).
|
||||||
|
`monitor` reads Minecraft's log volume and queries its RCON, but starts fine
|
||||||
|
without either: a missing log file and a refused RCON dial are ordinary states
|
||||||
|
it reports rather than errors it dies on.
|
||||||
@@ -16,6 +16,7 @@
|
|||||||
| `files.${BASE_DOMAIN}` | `62-files.caddy` | static `/srv/files` (read-only, no-auth browse of the share) |
|
| `files.${BASE_DOMAIN}` | `62-files.caddy` | static `/srv/files` (read-only, no-auth browse of the share) |
|
||||||
| `distribution.${BASE_DOMAIN}` | `30-distribution.caddy` | static `/srv/distribution` (other repo) |
|
| `distribution.${BASE_DOMAIN}` | `30-distribution.caddy` | static `/srv/distribution` (other repo) |
|
||||||
| `generators.${BASE_DOMAIN}` | `70-generators.caddy` | `generators:80` (offline misode fork + data mirror) |
|
| `generators.${BASE_DOMAIN}` | `70-generators.caddy` | `generators:80` (offline misode fork + data mirror) |
|
||||||
|
| `monitor.${BASE_DOMAIN}` | `80-monitor.caddy` | `monitor:8080` (admin log stream + dashboard, **basic_auth**) |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -110,9 +111,40 @@ passes it through, so don't rewrite it anywhere in the chain.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Monitor — `monitor.${BASE_DOMAIN}` (admin only)
|
||||||
|
|
||||||
|
`reverse_proxy monitor:8080`, behind caddy `basic_auth` (bcrypt,
|
||||||
|
`MONITOR_USER` / `MONITOR_PASSWORD_HASH`). **The only vhost in the stack that is
|
||||||
|
gated at caddy**, because the service itself has no auth and publishes no port —
|
||||||
|
caddy is the sole way in. Per-host on/off via `MONITOR_ENABLED`.
|
||||||
|
|
||||||
|
Read-only: every route is a `GET`, and nothing mutates state.
|
||||||
|
|
||||||
|
- `GET /` — the whole UI (single HTML file embedded in the binary).
|
||||||
|
- `GET /api/log[?since=<seq>]` → `{lines[], seq, dropped, reset, total, tail{}}`.
|
||||||
|
Omit `since` for the full buffer without a spurious "dropped" notice.
|
||||||
|
`since` is a **sequence number, not a byte offset** — the log rotates on every
|
||||||
|
deploy. `reset: true` means discard what you have (fell behind, or the service
|
||||||
|
restarted and your cursor is in the future).
|
||||||
|
- `GET /api/summary` → all dashboard tiles in one response. Each part is
|
||||||
|
independently cached and degrades on its own; failures appear as sibling
|
||||||
|
`*_err` fields next to the last good value.
|
||||||
|
- `GET /healthz` → `ok`. Liveness only, deliberately not tied to the log file.
|
||||||
|
|
||||||
|
Full design + gotchas: `plan/23-monitor.md`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Internal (service → service, over mcnet)
|
## Internal (service → service, over mcnet)
|
||||||
|
|
||||||
- `minecraft` → `-javaagent authlib-injector=http://auth.${BASE_DOMAIN}/authlib-injector` (caddy mcnet alias → drasl).
|
- `minecraft` → `-javaagent authlib-injector=http://auth.${BASE_DOMAIN}/authlib-injector` (caddy mcnet alias → drasl).
|
||||||
- `nmsr` → `http://auth.${BASE_DOMAIN}` (mojang-compat session/textures/api).
|
- `nmsr` → `http://auth.${BASE_DOMAIN}` (mojang-compat session/textures/api).
|
||||||
- `mc-status` → `http://drasl:25585/drasl/api/v2` (`DRASL_API_URL`) for the roster; pings `MC_STATUS_TARGET` for live status.
|
- `mc-status` → `http://drasl:25585/drasl/api/v2` (`DRASL_API_URL`) for the roster; pings `MC_STATUS_TARGET` for live status.
|
||||||
- `mc-backup` → RCON `minecraft:25575`.
|
- `mc-backup` → RCON `minecraft:25575`.
|
||||||
|
- `monitor` → `http://mc-status:8080` (`/v2/status/java/*` + `/players`) for the
|
||||||
|
status and roster tiles, so it needs no Drasl credentials of its own; RCON
|
||||||
|
`minecraft:25575` for TPS, restricted to a compile-time allowlist
|
||||||
|
(`neoforge tps`, `list`) that no HTTP input can reach. Reads
|
||||||
|
`mc_data:/data:ro` (log + `statfs`) and `./backups:ro` (backup freshness).
|
||||||
|
- `uptime-kuma` → can probe `http://monitor:8080/healthz` directly, bypassing
|
||||||
|
caddy, so the monitor vhost needs no unauthenticated path carve-out.
|
||||||
Reference in new issue
Block a user