Files
HexaHost-GameCloud/docs/operations/backup-restore.md

3.7 KiB

Backup and restore

This runbook covers backing up and restoring HexaHost GameCloud control plane data. Game server world data is backed up separately via the in-product backup feature (stored in object storage).

Scope

Component Method Frequency
PostgreSQL pg_dump + WAL archiving Daily full; continuous WAL (recommended)
Redis RDB snapshot (BGSAVE) Hourly if non-transient keys matter
Object storage (MinIO/S3) Replication or mc mirror Continuous or daily
Traefik ACME Copy acme.json After each cert issue
.env.prod / secrets Encrypted off-site vault On change
WHMCS mappings DB export of mod_hexagamecloud_* Weekly

PostgreSQL backup

Logical dump (single host)

docker exec hexahost-gamecloud-prod-postgres-1 \
  pg_dump -U gamecloud -Fc gamecloud > "gamecloud-$(date +%F).dump"

Store dumps encrypted off-site. Retain at least 30 daily backups for production.

Restore from dump

  1. Stop API and worker to prevent writes:

    docker compose -f deploy/compose/compose.prod.yml stop api worker
    
  2. Drop and recreate database (destructive):

    docker exec -i hexahost-gamecloud-prod-postgres-1 \
      psql -U gamecloud -c "DROP DATABASE gamecloud WITH (FORCE);"
    docker exec -i hexahost-gamecloud-prod-postgres-1 \
      psql -U gamecloud -c "CREATE DATABASE gamecloud OWNER gamecloud;"
    
  3. Restore:

    cat gamecloud-2026-07-01.dump | docker exec -i hexahost-gamecloud-prod-postgres-1 \
      pg_restore -U gamecloud -d gamecloud --no-owner --role=gamecloud
    
  4. Start services and verify:

    docker compose -f deploy/compose/compose.prod.yml start api worker
    curl -s https://api.example.net/api/v1/health/live
    

Point-in-time recovery (PITR)

For managed PostgreSQL or self-hosted WAL archiving, enable archive_mode and restore to a timestamp before the incident. Document your WAL retention in disaster-recovery.

Redis backup

Redis holds job queues and ephemeral cache. For queue durability during maintenance:

docker exec hexahost-gamecloud-prod-redis-1 redis-cli BGSAVE
docker cp hexahost-gamecloud-prod-redis-1:/data/dump.rdb ./redis-$(date +%F).rdb

Restore by stopping Redis, replacing dump.rdb, and restarting. Expect in-flight jobs to be reprocessed or dead-lettered.

Object storage backup

mc alias set prod http://minio:9000 "$S3_ACCESS_KEY" "$S3_SECRET_KEY"
mc mirror prod/gamecloud /backup/gamecloud-$(date +%F)/

Verify checksums on a sample of backup objects (backups/*/manifest.json).

Game server backups (tenant data)

Customer-initiated backups are stored under s3://gamecloud/backups/{serverId}/. Restore through the panel or admin API — not via PostgreSQL restore alone.

Pre-restore checklist

  • Incident ticket with restore target time documented
  • WHMCS placed in maintenance mode if billing sync is active
  • Node agents will reconnect automatically after API restore
  • DNS records remain valid (no restore needed for RFC2136 provider state in DB)

Verification after restore

  1. Admin login and random server detail page
  2. WHMCS addon dashboard health (Addons → HexaHost GameCloud)
  3. Worker queue processing (failed jobs near zero)
  4. Sample backup download from object storage

Automation

Integrate dumps with your existing backup tool (restic, Borg, Velero). Ansible playbooks in deploy/ansible/ provision hosts; schedule backups via cron or systemd timers on cp-db or the compose host.