JB/ROLLBACK_RUNBOOK.md

1.4 KiB

JobsBoard — Production Rollback Runbook

Emergency Rollback Triggers

  • Severe database migration failure or data inconsistency.
  • Critical unhandled exceptions affecting user authentication or job application pipeline.
  • Production error rate > 2% or 5xx response spike.

Rollback Procedure

Step 1: Revert Code to Prior Release

git checkout <PREVIOUS_STABLE_COMMIT_HASH>
cd web
npm ci
npm run build

Step 2: Restore Database Snapshot (If Schema Was Corrupted)

For PostgreSQL:

# Terminate existing connections
podman exec -i jobsboard-postgres psql -U postgres -d postgres -c \
  "SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = 'jobsboard';"

# Drop and restore from pre-deployment dump
podman exec -i jobsboard-postgres psql -U postgres -d postgres -c "DROP DATABASE jobsboard;"
podman exec -i jobsboard-postgres psql -U postgres -d postgres -c "CREATE DATABASE jobsboard;"
cat /backups/jobsboard_pre_deploy.sql | podman exec -i jobsboard-postgres psql -U postgres -d jobsboard

Step 3: Restart Services

pm2 restart jobsboard-web

Step 4: Verification

Execute health checks and baseline count audit:

curl -s http://localhost:3000/api/health | jq .
node -e '
const { PrismaClient } = require("@prisma/client");
const p = new PrismaClient();
p.job.count().then(c => { console.log("Jobs:", c); p.$disconnect(); });
'