ulicraft-server-v1/plan/23-monitor.md

23 — Monitor (monitor.${BASE_DOMAIN})

Admin-only, read-only monitoring page: realtime latest.log stream with autoscroll + filter, plus a small dashboard of live server facts. One new Go service (docker/monitor/), one caddy vhost behind HTTP basic auth, no published port, no actions of any kind in the UI.

Scope, stated as constraints

These were settled by design interview; they are the constraints, not preferences.

1. Admin-only. Nothing here is player-facing. Player-facing status already has two homes: the landing's ServerCard (15-mc-status.md) and Uptime Kuma (10-uptime-kuma.md). Because the audience is one person, log lines are never sanitized — they carry IPs, chat and admin commands verbatim.
2. Read-only. No controls. No console, no start/stop, no command input, no kick/ban. The UI issues GETs and nothing else. This is a design rule, not a v1 limitation — see RCON for how it survives contact with TPS.
3. latest.log only. Not whole-stack container logs. This is what kills the docker-socket requirement outright (see Rejected).
4. Delay is acceptable. Explicitly accepted at design time, and it is what makes polling viable over SSE.
5. Unreachable except through the subdomain. No host port, no LAN access, no direct container access from outside mcnet.

The one fact that drives every decision

The monitor never needs to reach outside mcnet, and nothing outside mcnet needs to reach it except caddy. So it publishes no port at all — same as drasl, nmsr, filestash and mc-status. Constraint 5 then costs zero firewall work: there is no host-level listener to firewall. nmap against the host still shows only 25565, 24454/udp, 80 and 443.

Everything else follows: no DOCKER-USER iptables chain, no bind-address env var, no ufw rule that Docker would silently bypass anyway.

Architecture

browser ── https://monitor.${BASE_DOMAIN}
   └► host nginx (TLS, own LE cert)
        └► caddy  80-monitor.caddy  ── basic_auth (bcrypt) ──► monitor:8080
                                                                 │
   monitor container:                                            │
     mc_data:/data:ro       ← tail /data/logs/latest.log  ────────┤
     ./backups:/backups:ro  ← newest archive stat (phase 2)       │
     RCON → minecraft:25575 ← allowlisted queries only (phase 3)  │
     HTTP → mc-status:8080  ← status + roster reuse (phase 2)     │
     //go:embed ui/         ← single index.html, no build step ───┘

Piece	Where
Service source	docker/monitor/ (main.go, go.mod, Dockerfile, ui/index.html)
Image	ulicraft/monitor:local, two-stage → scratch (same shape as mc-status)
Compose service	monitor, networks: [mcnet], no ports:
Ingress	caddy/conf.d/80-monitor.caddy
TLS / vhost	nginx/ulicraft-caddy.conf.tmpl + LE_SUBDOMAINS
Per-host toggle	MONITOR_ENABLED (mirrors the FILES_BROWSE on/off pattern)

Log pipeline

Transport: polling, not SSE

GET /api/log?since=<seq> → {lines: [...], seq: N, dropped: M}, client polls every ~2s.

Why not SSE. The request chain is host-nginx → caddy → app. nginx buffers proxied responses by default, so an EventSource stalls until the buffer fills unless the vhost carries proxy_buffering off;. That vhost is the per-host untracked file on cochi (see cochi host-local divergence), so the failure would appear only on prod and not reproduce locally. Constraint 4 means polling costs nothing, and it deletes the whole bug class. WebSocket was rejected for the same reason plus an upgrade-header hop it doesn't need.

Cursor is a sequence number, NOT a byte offset

The server keeps an in-memory ring buffer (5000 lines) and numbers lines monotonically. since is that number.

• A byte offset stops meaning anything the moment the file rotates, and itzg rotates latest.log → YYYY-MM-DD-N.log.gz on every server restart — i.e. on every deploy. Offsets are wrong routinely, not rarely.
• dropped > 0 tells the client it fell behind the ring and must re-request the full buffer rather than silently skip lines. Without this the UI lies by omission after any burst (mod crash spam, world load).

Rotation handling

The tailer stats the path each tick and reopens on inode change or size shrink. Both happen on every deploy. A tailer that only follows an open fd goes permanently silent after the first restart while still returning HTTP 200 — the classic version of "the monitor says everything is fine."

Cold start

On boot, read the tail of /data/logs/latest.log to prime the ring. That yields 5000 lines of history immediately with no database, no persistence, no state volume. A monitor restart costs nothing that isn't still on disk. Session history (joins/leaves) is derived from the same buffer, not stored.

Dashboard tiles

Everything below is available with no new privileges. Uptime Kuma already owns up/down and uptime history — do not rebuild that here.

Tile	Source	Phase
Server online + MOTD + icon	mc-status:8080/v2/status/java/… over mcnet	2
Players online (names + NMSR heads)	same, players.list[]	2
Registered players	mc-status:8080/players (Drasl-backed)	2
Disk free on the world volume	statfs on /data	2
Backup freshness	newest file in ./backups (:ro mount)	2
Join/leave session history	parsed from the buffer already tailed	2
TPS / MSPT	RCON, see below	3

Deliberately excluded: service reachability probes (Kuma's job), world-size walk (expensive, low value, cache-or-nothing), container health (needs the docker socket).

RCON — the one credential, and why it doesn't break constraint 2

TPS is the number you actually want when someone says "the server is lagging", and the only source without adding a 165th mod is an RCON query. So the monitor holds RCON_PASSWORD and dials minecraft:25575.

Constraint 2 survives because the UI never sends a command: the poller runs a compile-time allowlist (neoforge tps, list) on a 30s timer. There is no code path from an HTTP request to an RCON string. Hardcode the allowlist so a future refactor can't quietly turn this into a web console.

• The command is /neoforge tps on NeoForge 1.21.1, not /forge tps. Verify against the running server before wiring it, and treat an unknown-command reply as "TPS unavailable" rather than letting it kill the poll loop.
• Alternative considered: the spark mod. Better data (real profiling), but a 165th jar into a pack already over its stated ceiling, plus its own web endpoint to secure. Rejected for now; still the upgrade path if RCON TPS proves too coarse.

Auth + ingress

Basic auth lives in caddy, not host nginx: caddy/conf.d/*.caddy is in git, the nginx vhosts are per-host untracked drift on cochi.

# Monitor — admin-only log stream + dashboard (monitor service). Read-only.
http://monitor.{$BASE_DOMAIN} {
	basic_auth {
		{$MONITOR_USER} {$MONITOR_PASSWORD_HASH}
	}
	reverse_proxy monitor:8080
}

• caddy basic_auth needs a bcrypt hash (caddy hash-password). Note this is the opposite of the filestash trap in 20-files.md, where bcrypt is the hash that silently fails and $6$ is required. The two services take different formats; don't copy a hash between them.
• One user, one password (MONITOR_USER / MONITOR_PASSWORD_HASH in .env).
• Uptime Kuma does not need an auth exemption. Kuma is on mcnet, so it probes http://monitor:8080/healthz directly and never traverses caddy. The whole vhost stays gated with no public path carve-out.

Rejected ingress alternatives

• Apex path /monitor/* (the mc-status precedent — no DNS, no cert). Serving a JS app from a subpath is exactly the failure class 20-files.md documents for filestash. Fixable with relative asset paths, but it's a permanent footgun tax to save one DNS record.
• NetBird / VPN-only, no public listener. Considered and dropped. Worth recording why it was attractive: zero auth surface, logs never touch the internet. And why it was dropped: NetBird gates reachability, not identity — any peer the ACL admits gets the full log stream with no login, so the security model silently degrades the moment a second device or person joins that network.
• SSH tunnel to 127.0.0.1:9090. Most secure option available. Dropped because it defeats checking the server from a phone.

Rejected: whole-stack logs via the docker socket

Following caddy/drasl/filestash/mc-backup logs is genuinely where a lot of the debugging value lives (deploy failures surface in caddy and drasl, not in latest.log). It was dropped because constraint 3 limits scope to latest.log, and that removes the need for the socket entirely.

Recorded so the tradeoff isn't relitigated blind:

Mounting /var/run/docker.sock into a container is equivalent to granting that container root on the host, and the :ro flag does not prevent this — it makes the socket file read-only while the Docker API behind it still accepts container-create calls. The mitigation, had we needed it, is tecnativa/docker-socket-proxy restricted to GET containers+logs.

As designed, the monitor's only mount is mc_data:/data:ro (plus ./backups:ro in phase 2), and its only secret is RCON_PASSWORD in phase 3.

UI

Single index.html, //go:embed-ed into the binary. Vanilla JS + CSS, no build step, no pnpm in the deploy path, image stays a two-stage scratch build. Copy the landing's CSS custom properties so it doesn't look alien.

Behaviour:

• Autoscroll sticks to bottom only while already at the bottom. Scroll up and it stops fighting you; a "jump to live" button returns.
• Filter is plain substring, case-insensitive, client-side over the loaded buffer. /re/ syntax opts into regex.
• Level chips (ERROR / WARN / INFO) as quick toggles — log4j lines are [HH:mm:ss] [thread/LEVEL], so this is a cheap parse.
• ERROR red, WARN amber, everything else default.

Rejected: an Astro page (reuses landing components but adds a second pnpm build to update-all.sh), and caddy-served static assets (UI and API then version-skew independently).

Deploy surface

New/changed files:

• docker/monitor/{main.go,go.mod,Dockerfile,ui/index.html}
• docker-compose.yml — monitor service (no ports:)
• caddy/conf.d/80-monitor.caddy
• nginx/ulicraft-caddy.conf.tmpl — one server block + the :80 redirect name
• .env.example — MONITOR_USER, MONITOR_PASSWORD_HASH, MONITOR_ENABLED, LE_SUBDOMAINS += monitor
• plan/00-overview.md, plan/19-routes.md — index + route table

Gotchas:

• update-all.sh rolling mode cannot see caddy conf changes. It has no pre-pull baseline for caddy/conf.d/*.caddy, so the first deploy needs --hard — a full down/up, i.e. Minecraft world downtime. Later monitor-only changes are rolling (up -d --build monitor).
• DNS + cert are the long pole and live outside this repo. monitor. needs an A record, an LE_SUBDOMAINS entry, and tooling/issue-letsencrypt.sh. Nothing works until that lands; do it first.
• Per-host. cochi and oxps diverge — gate with MONITOR_ENABLED rather than assuming both hosts want it.

Build phases

Ship the thin slice first. Every hard part of this project is plumbing — new subdomain, cert, nginx vhost, basic auth through two proxies, mc_data:ro, tail-across-rotation. None of it is in the tiles. Once a password-gated page shows raw log lines, everything after is a small additive commit with a rolling one-container restart.

Phase 1 — thin slice: logs only

• DNS A record for monitor.${BASE_DOMAIN}; add to LE_SUBDOMAINS; issue cert.
• docker/monitor/ — Go service: ring buffer, tailer (inode/shrink reopen), cold-start backfill, GET /api/log?since=, GET /healthz, embedded UI.
• ui/index.html — log pane, sticky autoscroll, filter, level chips.
• compose service (no ports:, mc_data:/data:ro).
• caddy/conf.d/80-monitor.caddy + MONITOR_USER/MONITOR_PASSWORD_HASH.
• nginx vhost + MONITOR_ENABLED gate; render + reload.
• Deploy with --hard (world downtime). Verify below.

Phase 2 — tiles

• ./backups:/backups:ro mount; backup-freshness + disk-free tiles.
• mc-status reuse: online/MOTD/icon, players online w/ NMSR heads, registered count.
• Join/leave session history parsed from the buffer.

Phase 3 — TPS

• RCON client, compile-time allowlist, 30s poll.
• Confirm /neoforge tps is the right command on the running server.

Follow-ups

• Uptime Kuma HTTP monitor against http://monitor:8080/healthz (internal, unauthenticated by construction).
• Decide whether "search all history" (server-side grep over rotated .log.gz) is missed. Deferred from v1 on purpose: latest.log on a 164-mod server is mostly boot spam, and the real workflow is watching the last few hundred lines filtered by a player name or ERROR.

Verification

A 200 on / proves the server is up, not that the page works (see spa-curl-200-is-not-verification). After deploy, check all of:

# auth actually gates it (expect 401)
curl -s -o /dev/null -w '%{http_code}\n' https://monitor.${BASE_DOMAIN}/

# with creds, expect 200 + a non-empty buffer
curl -s -u "$MONITOR_USER:$MONITOR_PASSWORD" \
  "https://monitor.${BASE_DOMAIN}/api/log?since=0" | jq '.seq, (.lines|length)'

# the port is NOT reachable from outside the compose network
nmap -Pn -p 8080 <host-public-ip>        # expect closed/filtered
docker compose port monitor 8080          # expect an error: no published port

Then the one that actually matters, because it is the failure this design is most likely to hit: restart the Minecraft server and confirm the stream keeps moving. That is the rotation path, it fires on every deploy, and a broken tailer stays HTTP-200 while going silent.