Backup & Restore Guide

This document covers backup strategies, procedures, and disaster recovery for Tenanto.

Table of Contents

1. Overview
2. What to Backup
3. Database Backups
4. File Storage Backups
5. Configuration Backups
6. Automated Backup Scripts
7. Restore Procedures
8. Disaster Recovery
9. Testing Backups

Overview

Backup Strategy: 3-2-1 Rule

Follow the 3-2-1 backup rule:

• 3 copies of your data
• 2 different storage media
• 1 offsite copy

Backup Types

What to Backup

Critical Data

Not Needed in Backup

• vendor/ - Reinstall via composer install
• node_modules/ - Reinstall via npm ci
• storage/logs/ - Ephemeral, not critical
• storage/framework/cache/ - Regenerated automatically
• public/build/ - Rebuilt via npm run build

Database Backups

Manual Backup

# Full database backup
pg_dump -U tenanto -h localhost -Fc tenanto > backup_$(date +%Y%m%d_%H%M%S).dump

# Plain SQL format (larger but readable)
pg_dump -U tenanto -h localhost tenanto > backup_$(date +%Y%m%d_%H%M%S).sql

# Compressed plain SQL
pg_dump -U tenanto -h localhost tenanto | gzip > backup_$(date +%Y%m%d_%H%M%S).sql.gz

Schema Only (for development)

pg_dump -U tenanto -h localhost --schema-only tenanto > schema.sql

Data Only

pg_dump -U tenanto -h localhost --data-only tenanto > data.sql

Docker Environment

# From host machine
docker compose exec -T db pg_dump -U tenanto tenanto > backup.sql

# Compressed
docker compose exec -T db pg_dump -U tenanto tenanto | gzip > backup.sql.gz

Backup Specific Tables

# Backup only tenants and users (for migration)
pg_dump -U tenanto -h localhost -t tenants -t users tenanto > core_tables.sql

File Storage Backups

Local Storage

# Backup user uploads
tar -czvf uploads_$(date +%Y%m%d).tar.gz storage/app/public/

# Incremental backup with rsync
rsync -avz --delete storage/app/public/ /backup/files/

S3/Object Storage

If using S3, configure cross-region replication or use AWS Backup:

# Sync to backup bucket
aws s3 sync s3://tenanto-production s3://tenanto-backup --storage-class STANDARD_IA

Configuration Backups

Encrypt Sensitive Config

# Backup .env with encryption
gpg --symmetric --cipher-algo AES256 .env -o env_backup_$(date +%Y%m%d).gpg

# Decrypt
gpg -d env_backup_20250101.gpg > .env.restored

Version Control Configs

Keep deployment configs in version control (without secrets):

• docker-compose.yml
• nginx/*.conf
• .env.example

Automated Backup Scripts

Daily Backup Script

Create /opt/scripts/tenanto-backup.sh:

#!/bin/bash
set -e

# Configuration
BACKUP_DIR="/var/backups/tenanto"
RETENTION_DAYS=14
DB_NAME="tenanto"
DB_USER="tenanto"
APP_DIR="/var/www/tenanto"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
LOG_FILE="/var/log/tenanto-backup.log"

# Ensure backup directory exists
mkdir -p "$BACKUP_DIR"/{database,files,config}

log() {
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
}

log "Starting backup..."

# 1. Database backup
log "Backing up database..."
pg_dump -U "$DB_USER" -Fc "$DB_NAME" > "$BACKUP_DIR/database/db_$TIMESTAMP.dump"
if [ $? -eq 0 ]; then
    log "Database backup completed: db_$TIMESTAMP.dump"
else
    log "ERROR: Database backup failed!"
    exit 1
fi

# 2. File storage backup
log "Backing up uploads..."
tar -czf "$BACKUP_DIR/files/uploads_$TIMESTAMP.tar.gz" \
    -C "$APP_DIR" storage/app/public 2>/dev/null || true
log "Files backup completed: uploads_$TIMESTAMP.tar.gz"

# 3. Configuration backup (encrypted)
log "Backing up configuration..."
if [ -f "$APP_DIR/.env" ]; then
    # Use a passphrase from environment or file
    echo "$BACKUP_ENCRYPTION_KEY" | gpg --batch --yes --passphrase-fd 0 \
        --symmetric --cipher-algo AES256 \
        "$APP_DIR/.env" -o "$BACKUP_DIR/config/env_$TIMESTAMP.gpg"
    log "Config backup completed: env_$TIMESTAMP.gpg"
fi

# 4. Cleanup old backups
log "Cleaning up old backups..."
find "$BACKUP_DIR/database" -name "*.dump" -mtime +$RETENTION_DAYS -delete
find "$BACKUP_DIR/files" -name "*.tar.gz" -mtime +$RETENTION_DAYS -delete
find "$BACKUP_DIR/config" -name "*.gpg" -mtime +$RETENTION_DAYS -delete

# 5. Calculate backup sizes
DB_SIZE=$(du -sh "$BACKUP_DIR/database/db_$TIMESTAMP.dump" | cut -f1)
FILES_SIZE=$(du -sh "$BACKUP_DIR/files/uploads_$TIMESTAMP.tar.gz" 2>/dev/null | cut -f1 || echo "0")

log "Backup completed successfully!"
log "Database: $DB_SIZE | Files: $FILES_SIZE"

# Optional: Upload to offsite storage
# aws s3 cp "$BACKUP_DIR/database/db_$TIMESTAMP.dump" s3://tenanto-backups/database/
# aws s3 cp "$BACKUP_DIR/files/uploads_$TIMESTAMP.tar.gz" s3://tenanto-backups/files/

Cron Configuration

# Daily backup at 3 AM
0 3 * * * /opt/scripts/tenanto-backup.sh >> /var/log/tenanto-backup.log 2>&1

# Weekly full backup with longer retention (Sundays at 2 AM)
0 2 * * 0 /opt/scripts/tenanto-full-backup.sh >> /var/log/tenanto-backup.log 2>&1

Systemd Timer (Alternative to Cron)

/etc/systemd/system/tenanto-backup.timer:

[Unit]
Description=Daily Tenanto Backup

[Timer]
OnCalendar=*-*-* 03:00:00
Persistent=true

[Install]
WantedBy=timers.target

/etc/systemd/system/tenanto-backup.service:

[Unit]
Description=Tenanto Backup Service
After=postgresql.service

[Service]
Type=oneshot
ExecStart=/opt/scripts/tenanto-backup.sh
User=root
Environment=BACKUP_ENCRYPTION_KEY=your-secure-key

[Install]
WantedBy=multi-user.target

Enable:

sudo systemctl enable --now tenanto-backup.timer

Restore Procedures

Database Restore

Full Restore

# Stop application
php artisan down

# Drop and recreate database
sudo -u postgres psql -c "DROP DATABASE tenanto;"
sudo -u postgres psql -c "CREATE DATABASE tenanto OWNER tenanto;"

# Restore from custom format
pg_restore -U tenanto -d tenanto backup.dump

# Or from SQL format
psql -U tenanto -d tenanto < backup.sql

# Run any pending migrations
php artisan migrate --force

# Clear caches
php artisan cache:clear
php artisan config:cache
php artisan route:cache

# Bring application back up
php artisan up

Docker Environment Restore

# Stop containers
docker compose down

# Start only database
docker compose up -d db
sleep 5

# Restore database
cat backup.sql | docker compose exec -T db psql -U tenanto -d tenanto

# Start all services
docker compose up -d

Point-in-Time Recovery

For PostgreSQL with WAL archiving enabled:

# Restore base backup
pg_restore -U tenanto -d tenanto_restore base_backup.dump

# Apply WAL logs up to specific time
# (Requires PostgreSQL configured with archive_mode and recovery.conf)

File Storage Restore

# Extract uploads
tar -xzvf uploads_20250101.tar.gz -C /var/www/tenanto/

# Fix permissions
chown -R www-data:www-data /var/www/tenanto/storage
chmod -R 775 /var/www/tenanto/storage

# Recreate storage link if needed
php artisan storage:link

Configuration Restore

# Decrypt .env backup
gpg -d env_backup_20250101.gpg > .env

# Verify and update as needed (database credentials, etc.)
nano .env

# Regenerate caches
php artisan config:cache

Disaster Recovery

Recovery Time Objectives

Recovery Checklist

1. Assess Situation

• Identify what failed (database, server, storage)
• Determine data loss window
• Notify stakeholders

2. Provision Infrastructure

• Spin up new server/container
• Install dependencies (PHP, PostgreSQL, Redis, nginx)
• Configure firewall and SSL

3. Restore Data

• Restore database from latest backup
• Restore file uploads
• Restore configuration (.env)

4. Verify Application

• Run migrations if needed
• Clear and rebuild caches
• Restart queue workers
• Test critical user flows

5. Post-Recovery

• Update DNS if IP changed
• Verify monitoring and alerting
• Document incident
• Review and improve backup procedures

Runbook: Complete Server Recovery

# 1. Provision new server with required software
sudo apt update && sudo apt install -y \
    php8.4-fpm php8.4-cli php8.4-pgsql php8.4-redis \
    php8.4-bcmath php8.4-intl php8.4-gd php8.4-xml \
    php8.4-curl php8.4-zip php8.4-mbstring \
    nginx postgresql-16 redis-server supervisor

# 2. Clone application (or copy from your source)
cd /var/www
git clone https://your-git-host.com/your-org/tenanto.git tenanto
cd tenanto

# 3. Install dependencies
composer install --optimize-autoloader --no-dev
npm ci && npm run build

# 4. Restore configuration
gpg -d /backup/config/env_latest.gpg > .env

# 5. Restore database
pg_restore -U tenanto -d tenanto /backup/database/db_latest.dump

# 6. Restore uploads
tar -xzvf /backup/files/uploads_latest.tar.gz -C /var/www/tenanto/

# 7. Fix permissions
chown -R www-data:www-data /var/www/tenanto
chmod -R 755 /var/www/tenanto
chmod -R 775 storage bootstrap/cache

# 8. Rebuild caches
php artisan config:cache
php artisan route:cache
php artisan view:cache
php artisan storage:link

# 9. Restart services
sudo systemctl restart php8.4-fpm nginx
sudo supervisorctl restart tenanto-worker:*

# 10. Verify
curl -s http://localhost/health | jq

Testing Backups

Monthly Backup Verification

Critical: Untested backups are not backups!

# 1. Create test environment
docker compose -f docker-compose.test.yml up -d

# 2. Restore latest backup
docker compose -f docker-compose.test.yml exec db \
    pg_restore -U tenanto -d tenanto_test /backups/latest.dump

# 3. Run verification queries
docker compose -f docker-compose.test.yml exec db psql -U tenanto -d tenanto_test -c "
    SELECT COUNT(*) as tenants FROM tenants;
    SELECT COUNT(*) as users FROM users;
    SELECT COUNT(*) as projects FROM projects;
    SELECT COUNT(*) as tasks FROM tasks;
"

# 4. Test application
docker compose -f docker-compose.test.yml exec app php artisan migrate:status
docker compose -f docker-compose.test.yml exec app php artisan test --group=critical

# 5. Cleanup
docker compose -f docker-compose.test.yml down -v

Backup Integrity Checks

# Verify PostgreSQL backup integrity
pg_restore --list backup.dump > /dev/null && echo "Backup valid" || echo "Backup corrupted!"

# Verify tar archive
tar -tzf uploads.tar.gz > /dev/null && echo "Archive valid" || echo "Archive corrupted!"

# Verify GPG encrypted file
gpg --batch --decrypt --dry-run env_backup.gpg 2>/dev/null && echo "Encrypted file valid"

Monitoring Backup Health

Alert on Backup Failures

Add to your monitoring system:

# Example: Prometheus alert rule
- alert: BackupMissing
  expr: time() - backup_last_success_timestamp > 86400 * 2
  for: 1h
  labels:
    severity: critical
  annotations:
    summary: "Tenanto backup missing for more than 2 days"

Backup Report Script

#!/bin/bash
# Send daily backup status report

BACKUP_DIR="/var/backups/tenanto"

echo "=== Tenanto Backup Report $(date) ==="
echo ""
echo "Database backups:"
ls -lh "$BACKUP_DIR/database/" | tail -5
echo ""
echo "Total database backup size: $(du -sh "$BACKUP_DIR/database/" | cut -f1)"
echo ""
echo "File backups:"
ls -lh "$BACKUP_DIR/files/" | tail -5
echo ""
echo "Total file backup size: $(du -sh "$BACKUP_DIR/files/" | cut -f1)"

Related Documentation

• Deployment Guide - Production setup
• Security Guide - Security configuration
• Launch Checklist - Pre-launch requirements

Support

For backup and recovery assistance, contact the Tenanto operations team.