3.7 KiB
Backup and restore
This runbook covers backing up and restoring HexaHost GameCloud control plane data. Game server world data is backed up separately via the in-product backup feature (stored in object storage).
Scope
| Component | Method | Frequency |
|---|---|---|
| PostgreSQL | pg_dump + WAL archiving |
Daily full; continuous WAL (recommended) |
| Redis | RDB snapshot (BGSAVE) |
Hourly if non-transient keys matter |
| Object storage (MinIO/S3) | Replication or mc mirror |
Continuous or daily |
| Traefik ACME | Copy acme.json |
After each cert issue |
.env.prod / secrets |
Encrypted off-site vault | On change |
| WHMCS mappings | DB export of mod_hexagamecloud_* |
Weekly |
PostgreSQL backup
Logical dump (single host)
docker exec hexahost-gamecloud-prod-postgres-1 \
pg_dump -U gamecloud -Fc gamecloud > "gamecloud-$(date +%F).dump"
Store dumps encrypted off-site. Retain at least 30 daily backups for production.
Restore from dump
-
Stop API and worker to prevent writes:
docker compose -f deploy/compose/compose.prod.yml stop api worker -
Drop and recreate database (destructive):
docker exec -i hexahost-gamecloud-prod-postgres-1 \ psql -U gamecloud -c "DROP DATABASE gamecloud WITH (FORCE);" docker exec -i hexahost-gamecloud-prod-postgres-1 \ psql -U gamecloud -c "CREATE DATABASE gamecloud OWNER gamecloud;" -
Restore:
cat gamecloud-2026-07-01.dump | docker exec -i hexahost-gamecloud-prod-postgres-1 \ pg_restore -U gamecloud -d gamecloud --no-owner --role=gamecloud -
Start services and verify:
docker compose -f deploy/compose/compose.prod.yml start api worker curl -s https://api.example.net/api/v1/health/live
Point-in-time recovery (PITR)
For managed PostgreSQL or self-hosted WAL archiving, enable archive_mode and restore to a timestamp before the incident. Document your WAL retention in disaster-recovery.
Redis backup
Redis holds job queues and ephemeral cache. For queue durability during maintenance:
docker exec hexahost-gamecloud-prod-redis-1 redis-cli BGSAVE
docker cp hexahost-gamecloud-prod-redis-1:/data/dump.rdb ./redis-$(date +%F).rdb
Restore by stopping Redis, replacing dump.rdb, and restarting. Expect in-flight jobs to be reprocessed or dead-lettered.
Object storage backup
mc alias set prod http://minio:9000 "$S3_ACCESS_KEY" "$S3_SECRET_KEY"
mc mirror prod/gamecloud /backup/gamecloud-$(date +%F)/
Verify checksums on a sample of backup objects (backups/*/manifest.json).
Game server backups (tenant data)
Customer-initiated backups are stored under s3://gamecloud/backups/{serverId}/. Restore through the panel or admin API — not via PostgreSQL restore alone.
Pre-restore checklist
- Incident ticket with restore target time documented
- WHMCS placed in maintenance mode if billing sync is active
- Node agents will reconnect automatically after API restore
- DNS records remain valid (no restore needed for RFC2136 provider state in DB)
Verification after restore
- Admin login and random server detail page
- WHMCS addon dashboard health (
Addons → HexaHost GameCloud) - Worker queue processing (
failedjobs near zero) - Sample backup download from object storage
Automation
Integrate dumps with your existing backup tool (restic, Borg, Velero). Ansible playbooks in deploy/ansible/ provision hosts; schedule backups via cron or systemd timers on cp-db or the compose host.