Files
collective-lists/infra/scripts/deploy-erosi.sh
Oier Bravo Urtasun ae4fd45b99 deploy(prod): pre-deploy DB backup + paired code/DB rollback skill
Every `just deploy` now takes a pg_dumpall to
/opt/colectivo/backups/predeploy-<ts>-<sha>.sql.gz BEFORE rebuilding the
app or applying migrations, aborts the deploy if the dump is < 1 KiB,
and (on success) appends `<iso-ts> \t <sha> \t <backup-file>` to
/opt/colectivo/.deploys.log on ambrosio. Keeps the newest 10 backups
(prunes older predeploy-* files).

New `infra/scripts/rollback-erosi.sh` reads .deploys.log and pairs a
code rollback with a DB restore atomically:

  just rollback-list          # show recent deploys
  just rollback               # roll back to N-1
  just rollback-to <sha>      # roll back to a specific deploy
  just rollback-code          # roll back code only, keep current DB

Rollback safety:
- 5-second abort window.
- Verifies the target SHA exists locally + the backup is still on prod
  (warns if it's been pruned past the 10-deploy window).
- Uses a temporary git worktree so the user's working tree isn't
  disturbed.
- Stops app/auth/rest/realtime/storage before the gunzip|psql restore.
- Rebuilds the app image at the rolled-back SHA with the same GIT_SHA
  build-arg path the deploy uses (so __APP_COMMIT__ in the bundle
  matches the running code).
- Does NOT write a new .deploys.log entry — rollback is intentionally
  not a deploy event; the next `just deploy` is from current HEAD.
- Storage volume (/var/lib/storage user uploads) is NOT rolled back.

Justfile `deploy` recipe repointed from the stale `deploy.sh` (which
referenced GHCR pull) to `deploy-erosi.sh` (the active prod path).

New project-scoped skill: `.claude/skills/deploy/SKILL.md` documents the
flow + safety boundaries for Claude-assisted invocations. `.gitignore`
keeps `.claude/settings.local.json` ignored but tracks `.claude/skills/`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 14:57:25 +02:00

221 lines
8.9 KiB
Bash
Executable File

#!/usr/bin/env bash
# Deploy the full stack to ambrosio.
#
# Usage: infra/scripts/deploy-erosi.sh
# Env: DEPLOY_HOST=ambrosio (override if needed)
# DEPLOY_PATH=/opt/colectivo
#
# First run:
# 1. rsyncs repo → ambrosio:/opt/colectivo/
# 2. if .env doesn't exist on server, generates secrets and writes it with
# placeholder values the operator must fill in (external Keycloak URL,
# Keycloak client secret)
# 3. docker compose build + up -d
# 4. applies DB migrations
#
# Internal edge is a Caddy container listening on host port 3000 — the
# external proxy (TLS terminator, off-stack) forwards plain HTTP to
# ambrosio:3000. This script never touches the host reverse-proxy config —
# it only manages the Docker stack.
#
# Subsequent runs: rsync + rebuild app + rolling restart.
#
# Pre-deploy safety (every run, before any code/migration changes):
# * pg_dumpall to /opt/colectivo/backups/predeploy-<ts>-<sha>.sql.gz
# * Keep newest 10 backups, prune older.
# * On success, append a line to /opt/colectivo/.deploys.log:
# <iso-timestamp> <git-sha> <backup-filename>
# The rollback script (infra/scripts/rollback-erosi.sh) reads .deploys.log
# to pair (sha, backup) when rolling back code + DB together.
set -euo pipefail
DEPLOY_HOST="${DEPLOY_HOST:-ambrosio}"
DEPLOY_PATH="${DEPLOY_PATH:-/opt/colectivo}"
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
# Fase 14.3.5 — capture the short git sha on the deploy host (this
# machine) so it can be baked into the built app bundle as
# `__APP_COMMIT__`. We do it here rather than on ambrosio because we
# rsync without `.git`, so the remote can't run `git rev-parse`.
GIT_SHA="$(cd "$REPO_ROOT" && git rev-parse --short HEAD 2>/dev/null || echo dev)"
echo "==> Deploying to $DEPLOY_HOST:$DEPLOY_PATH (commit $GIT_SHA)"
# ── 1. Ensure target dir exists ────────────────────────────────────────────
ssh "$DEPLOY_HOST" "sudo mkdir -p $DEPLOY_PATH && sudo chown \$(id -u):\$(id -g) $DEPLOY_PATH"
# ── 2. rsync repo (excluding node_modules / build artefacts / local env) ──
rsync -az --delete \
--exclude '.git' \
--exclude 'node_modules' \
--exclude '.svelte-kit' \
--exclude 'apps/web/build' \
--exclude 'apps/web/.vite' \
--exclude '**/*.log' \
--exclude '**/.env' \
--exclude '**/.env.development' \
--exclude '**/.env.local' \
--exclude 'lighthouse-report.html' \
--exclude 'coverage' \
--exclude 'test-results' \
--exclude 'playwright-report' \
"$REPO_ROOT/" "$DEPLOY_HOST:$DEPLOY_PATH/"
# ── 3. Bootstrap secrets on first run; sanity-check placeholders every run ─
ssh "$DEPLOY_HOST" bash -s << 'REMOTE'
set -euo pipefail
cd /opt/colectivo
if [ ! -f .env ]; then
echo "--- First deploy: generating secrets"
# JWT triplet (anon + service_role)
JWT_OUT=$(bash infra/scripts/rotate-jwt.sh)
SUPABASE_JWT_SECRET=$(echo "$JWT_OUT" | awk -F= '/^SUPABASE_JWT_SECRET=/{print substr($0, index($0,$2))}')
PUBLIC_SUPABASE_ANON_KEY=$(echo "$JWT_OUT" | awk -F= '/^PUBLIC_SUPABASE_ANON_KEY=/{print substr($0, index($0,$2))}')
SUPABASE_SERVICE_ROLE_KEY=$(echo "$JWT_OUT" | awk -F= '/^SUPABASE_SERVICE_ROLE_KEY=/{print substr($0, index($0,$2))}')
# Postgres + Realtime secrets
POSTGRES_PASSWORD=$(openssl rand -hex 24)
REALTIME_ENC_KEY=$(openssl rand -hex 8) # 16 chars
REALTIME_SECRET_KEY_BASE=$(openssl rand -hex 48) # 96 chars
cat > .env <<EOF
# Generated on $(date -u +%Y-%m-%dT%H:%M:%SZ). Keep this file mode 600.
PUBLIC_APP_URL=https://erosi.limonia.net
PUBLIC_SUPABASE_URL=https://erosi.limonia.net
# External Keycloak — FILL THESE IN before the stack will boot usefully.
PUBLIC_KEYCLOAK_URL=FILL_IN_KEYCLOAK_URL
PUBLIC_KEYCLOAK_REALM=colectivo
PUBLIC_KEYCLOAK_CLIENT_ID=colectivo-web
KEYCLOAK_CLIENT_SECRET=FILL_IN_CLIENT_SECRET
POSTGRES_PASSWORD=$POSTGRES_PASSWORD
SUPABASE_JWT_SECRET=$SUPABASE_JWT_SECRET
PUBLIC_SUPABASE_ANON_KEY=$PUBLIC_SUPABASE_ANON_KEY
SUPABASE_SERVICE_ROLE_KEY=$SUPABASE_SERVICE_ROLE_KEY
REALTIME_ENC_KEY=$REALTIME_ENC_KEY
REALTIME_SECRET_KEY_BASE=$REALTIME_SECRET_KEY_BASE
EOF
chmod 600 .env
echo "--- Wrote .env (mode 600)."
echo " Fill in: PUBLIC_KEYCLOAK_URL, KEYCLOAK_CLIENT_SECRET"
echo " then re-run this script."
exit 0
fi
# Every deploy: fail fast if the operator didn't finish the .env.
if grep -q '^PUBLIC_KEYCLOAK_URL=FILL_IN_' .env \
|| grep -q '^KEYCLOAK_CLIENT_SECRET=FILL_IN_' .env; then
echo "ERROR: /opt/colectivo/.env still contains FILL_IN_ placeholders." >&2
echo " Fill in PUBLIC_KEYCLOAK_URL, KEYCLOAK_CLIENT_SECRET and retry." >&2
exit 1
fi
REMOTE
# ── 3b. Pre-deploy DB backup ──────────────────────────────────────────────
# Always taken BEFORE the new code/migrations land, so a paired rollback
# can restore both. Stored on ambrosio at /opt/colectivo/backups/. Keep
# newest 10; older ones get pruned.
ssh "$DEPLOY_HOST" env GIT_SHA="$GIT_SHA" bash -s << 'REMOTE'
set -euo pipefail
cd /opt/colectivo
: "${GIT_SHA:=dev}"
mkdir -p backups
BACKUP_TS=$(date -u +"%Y%m%dT%H%M%SZ")
BACKUP_FILE="backups/predeploy-${BACKUP_TS}-${GIT_SHA}.sql.gz"
echo "--- Pre-deploy backup → $BACKUP_FILE"
docker compose --env-file .env -f infra/docker-compose.erosi.yml exec -T db \
pg_dumpall -U postgres --clean --if-exists </dev/null \
| gzip > "$BACKUP_FILE"
size=$(stat -c%s "$BACKUP_FILE")
if [ "$size" -lt 1024 ]; then
echo "ERROR: backup is suspiciously small ($size bytes). Aborting deploy." >&2
rm -f "$BACKUP_FILE"
exit 1
fi
echo " ok ($(numfmt --to=iec --suffix=B "$size" 2>/dev/null || echo "$size bytes"))"
# Prune: keep newest 10 predeploy backups.
ls -1t backups/predeploy-*.sql.gz 2>/dev/null | tail -n +11 | xargs -r rm -f
# Stash filename for step 4 to record once everything succeeds.
echo "$BACKUP_FILE" > /tmp/.colectivo-last-backup
REMOTE
# ── 4. Build + bring up the stack ─────────────────────────────────────────
# We forward GIT_SHA over the ssh env so the remote `docker compose build`
# can substitute it into the Dockerfile build-arg (compose reads it from
# the same shell, not from .env, so a transient export is enough). ssh
# doesn't accept VAR=value args directly, so we prefix `env`.
ssh "$DEPLOY_HOST" env GIT_SHA="$GIT_SHA" bash -s << 'REMOTE'
set -euo pipefail
cd /opt/colectivo
: "${GIT_SHA:=dev}"
echo "--- Building app image (commit $GIT_SHA)"
GIT_SHA="$GIT_SHA" docker compose --env-file .env -f infra/docker-compose.erosi.yml \
build --build-arg GIT_SHA="$GIT_SHA" app
echo "--- Bringing stack up (detached)"
GIT_SHA="$GIT_SHA" docker compose --env-file .env -f infra/docker-compose.erosi.yml up -d
echo "--- Waiting for db to be healthy"
for i in $(seq 1 60); do
if docker compose --env-file .env -f infra/docker-compose.erosi.yml ps db --format json | grep -q '"Health":"healthy"'; then
echo "db healthy"; break
fi
sleep 2
done
echo "--- Applying migrations"
# Every `docker compose exec -T` call below pipes in `</dev/null` (or the
# migration file for the apply step). This matters because the outer bash
# is reading its own script from stdin (`ssh ... bash -s << REMOTE`); a
# `docker compose exec -T` without its own stdin source drains the heredoc,
# silently eating the remainder of the script — observed as "first iteration
# runs, every subsequent migration vanishes from the log".
docker compose --env-file .env -f infra/docker-compose.erosi.yml exec -T db \
psql -U postgres -d postgres </dev/null \
-c "CREATE TABLE IF NOT EXISTS public._applied_migrations (filename TEXT PRIMARY KEY, applied_at TIMESTAMPTZ DEFAULT now());"
for f in supabase/migrations/*.sql; do
fn=$(basename "$f")
already=$(docker compose --env-file .env -f infra/docker-compose.erosi.yml exec -T db \
psql -U postgres -d postgres -tAq </dev/null \
-c "SELECT count(*) FROM public._applied_migrations WHERE filename='$fn'")
if [ "${already:-0}" -gt 0 ]; then
echo " skip $fn (applied)"
continue
fi
printf " apply %s ... " "$fn"
if docker compose --env-file .env -f infra/docker-compose.erosi.yml exec -T db \
psql -U postgres -d postgres -v ON_ERROR_STOP=1 -q < "$f"; then
docker compose --env-file .env -f infra/docker-compose.erosi.yml exec -T db \
psql -U postgres -d postgres </dev/null \
-c "INSERT INTO public._applied_migrations (filename) VALUES ('$fn') ON CONFLICT DO NOTHING;"
echo "ok"
else
echo "FAIL"; exit 1
fi
done
# Record the successful deploy in .deploys.log (read by rollback-erosi.sh).
backup_file=$(cat /tmp/.colectivo-last-backup 2>/dev/null || echo "")
deploy_ts=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
printf '%s\t%s\t%s\n' "$deploy_ts" "$GIT_SHA" "$backup_file" \
>> /opt/colectivo/.deploys.log
rm -f /tmp/.colectivo-last-backup
echo "--- Recorded deploy in /opt/colectivo/.deploys.log"
REMOTE
echo "==> Deploy complete."
echo " App: https://erosi.limonia.net"
echo " Keycloak: external — see PUBLIC_KEYCLOAK_URL in $DEPLOY_PATH/.env"