Enhance API with OIDC support, including login and callback endpoints. Update environment variables for OIDC configuration in .env.example. Add new features to the catalog service for listing software families, Minecraft versions, and deployment regions. Implement server management actions such as kill, delete, and update in the servers module. Integrate feature flags for maintenance mode in server operations. Update pnpm-lock.yaml with new dependencies and versions.

docs/operations/object-storage.md

@@ -0,0 +1,90 @@
+# Object storage operations
+
+HexaHost GameCloud stores backups, world exports, mod archives, and other large blobs in S3-compatible object storage. Development uses MinIO via `deploy/compose/compose.dev.yml`; production uses MinIO or an external S3 provider.
+
+## Environment variables
+
+| Variable | Description |
+|----------|-------------|
+| `S3_ENDPOINT` | Internal URL, e.g. `http://minio:9000` in compose |
+| `S3_REGION` | Region identifier (`us-east-1` for MinIO) |
+| `S3_BUCKET` | Primary bucket (default `gamecloud`) |
+| `S3_ACCESS_KEY` / `S3_SECRET_KEY` | Credentials — rotate regularly |
+| `S3_FORCE_PATH_STYLE` | `true` for MinIO; `false` for AWS S3 |
+
+## Production (MinIO in compose)
+
+The production stack in `deploy/compose/compose.prod.yml` runs MinIO on the internal network only — no public ports. API and worker reach it at `http://minio:9000`.
+
+Initial bucket creation is handled by `minio-init` on first deploy:
+
+```bash
+docker compose -f deploy/compose/compose.prod.yml --env-file .env.prod up minio-init
+```
+
+### Admin console access
+
+Do not expose the MinIO console to the public internet. Options:
+
+- SSH tunnel: `ssh -L 9001:127.0.0.1:9001 cp-01`
+- VPN-only route to the control plane management network
+- Separate MinIO deployment with IAM and audit logging
+
+## External S3 (AWS, Wasabi, Hetzner Object Storage)
+
+Point `S3_ENDPOINT` at the provider URL and set `S3_FORCE_PATH_STYLE=false` where appropriate. Create the bucket manually and apply a bucket policy that:
+
+- Denies public read/write
+- Allows only the GameCloud service principal (access key)
+- Enables versioning for backup objects
+- Optionally enforces SSE-S3 or SSE-KMS
+
+Remove the `minio` and `minio-init` services from compose when using external storage.
+
+## Object layout
+
+| Prefix | Content | Retention |
+|--------|---------|-----------|
+| `backups/{serverId}/` | Scheduled server backups | Per plan + [data retention policy](../security/data-retention.md) |
+| `worlds/{serverId}/` | World export ZIPs | 7 days after download link expiry |
+| `uploads/{serverId}/` | User file uploads (staging) | Cleaned after successful install |
+| `catalog/` | Cached mod/plugin artifacts | Managed by catalog sync jobs |
+
+## Health and monitoring
+
+MinIO health endpoint: `GET /minio/health/live`
+
+Monitor:
+
+- Bucket size growth (per tenant and total)
+- Failed PUT/GET rates from API logs
+- Disk usage on MinIO volume `hexahost-gamecloud-minio`
+
+## Backup of the storage layer
+
+For self-hosted MinIO:
+
+1. Enable bucket versioning
+2. Replicate to a second bucket or provider (MinIO site replication or `mc mirror`)
+3. Include object storage in [backup-restore](backup-restore.md) runbooks
+
+For external S3, rely on provider replication and lifecycle rules; document RPO/RTO in [disaster-recovery](disaster-recovery.md).
+
+## Troubleshooting
+
+### `NoSuchBucket`
+
+Run `minio-init` or create the bucket manually with the configured `S3_BUCKET` name.
+
+### Signature errors
+
+Verify clock sync (NTP) on API/worker hosts. Check `S3_FORCE_PATH_STYLE` matches the provider.
+
+### Slow backup uploads
+
+Check network between game nodes (backup source) and storage. Large backups stream from node-agent → API → S3; consider dedicated storage nodes for high-volume deployments.
+
+## Related
+
+- [Backup and restore](backup-restore.md)
+- [Secrets management](../security/secrets.md)