database-backups

数据库备份

实施全面、自动化的数据库备份策略，并测试恢复流程。

适用场景

• 部署新数据库，需要从一开始就制定备份计划。
• 需要为 PostgreSQL、MySQL 或 MongoDB 自动执行每晚或每小时备份。
• 希望将备份上传到兼容 S3 的对象存储，并设置保留策略。
• 正在编写或验证灾难恢复运行手册。

先决条件

• 已安装数据库客户端工具（pg_dump、mysqldump、mongodump）。
• 已安装 AWS CLI 或 restic，用于远程存储。
• 已配置 cron 或 systemd 定时器来安排任务。
• 已准备 S3 存储桶（或兼容 S3 的端点），用于异地备份。

备份类型

类型	描述	频率	用途
全量	完整的数据库副本	每周	还原基线
增量	自上次备份以来的变更	每天	减少存储空间和耗时
事务日志 / WAL	持续传输日志	持续	时间点恢复（PITR）
快照	存储级快照（EBS、ZFS）	每天	快速完整还原

PostgreSQL 备份

使用 pg_dump 进行逻辑备份

#!/bin/bash
# pg_backup.sh — PostgreSQL logical backup
set -euo pipefail

DB_NAME="mydb"
DB_USER="backup_user"
DB_HOST="localhost"
BACKUP_DIR="/backups/postgres"
DATE=$(date +%Y%m%d_%H%M%S)
FILENAME="${BACKUP_DIR}/${DB_NAME}_${DATE}.dump"

mkdir -p "$BACKUP_DIR"

# Custom compressed format (recommended for selective restore)
pg_dump -h "$DB_HOST" -U "$DB_USER" -Fc -Z6 "$DB_NAME" > "$FILENAME"

echo "[$(date)] PostgreSQL backup complete: $FILENAME ($(du -h "$FILENAME" | cut -f1))"

使用 pg_basebackup 进行物理备份

#!/bin/bash
# pg_basebackup.sh — PostgreSQL physical backup for PITR
set -euo pipefail

BACKUP_DIR="/backups/postgres/base_$(date +%Y%m%d)"
mkdir -p "$BACKUP_DIR"

pg_basebackup \
  -h localhost \
  -U replicator \
  -D "$BACKUP_DIR" \
  --wal-method=stream \
  --checkpoint=fast \
  --progress \
  --verbose

echo "[$(date)] Base backup complete: $BACKUP_DIR"

PostgreSQL 恢复

# Restore from custom-format dump
pg_restore -h localhost -U myapp -d mydb --clean --if-exists /backups/postgres/mydb_20250115_020000.dump

# Restore a single table
pg_restore -h localhost -U myapp -d mydb -t orders /backups/postgres/mydb_20250115_020000.dump

# Restore from plain SQL
psql -h localhost -U myapp -d mydb < /backups/postgres/mydb_20250115.sql

MySQL 备份

使用 mysqldump 进行逻辑备份

#!/bin/bash
# mysql_backup.sh — MySQL logical backup
set -euo pipefail

DB_NAME="mydb"
DB_USER="backup_user"
DB_PASS="${MYSQL_BACKUP_PASSWORD}"
BACKUP_DIR="/backups/mysql"
DATE=$(date +%Y%m%d_%H%M%S)
FILENAME="${BACKUP_DIR}/${DB_NAME}_${DATE}.sql.gz"

mkdir -p "$BACKUP_DIR"

mysqldump -u "$DB_USER" -p"$DB_PASS" \
  --single-transaction \
  --routines \
  --triggers \
  --events \
  "$DB_NAME" | gzip > "$FILENAME"

echo "[$(date)] MySQL backup complete: $FILENAME ($(du -h "$FILENAME" | cut -f1))"

使用 Percona XtraBackup 进行物理备份

#!/bin/bash
# xtrabackup.sh — MySQL physical backup
set -euo pipefail

BACKUP_DIR="/backups/mysql/full_$(date +%Y%m%d)"

xtrabackup --backup \
  --user=backup_user \
  --password="${MYSQL_BACKUP_PASSWORD}" \
  --target-dir="$BACKUP_DIR"

xtrabackup --prepare --target-dir="$BACKUP_DIR"

echo "[$(date)] XtraBackup complete: $BACKUP_DIR"

MySQL 恢复

# Restore from compressed mysqldump
gunzip < /backups/mysql/mydb_20250115_020000.sql.gz | mysql -u root -p mydb

# Restore from XtraBackup
sudo systemctl stop mysql
sudo rm -rf /var/lib/mysql/*
xtrabackup --move-back --target-dir=/backups/mysql/full_20250115
sudo chown -R mysql:mysql /var/lib/mysql
sudo systemctl start mysql

MongoDB 备份

使用 mongodump 进行逻辑备份

#!/bin/bash
# mongo_backup.sh — MongoDB backup
set -euo pipefail

MONGO_URI="mongodb://backup_user:${MONGO_BACKUP_PASSWORD}@localhost:27017"
BACKUP_DIR="/backups/mongodb"
DATE=$(date +%Y%m%d_%H%M%S)
TARGET="${BACKUP_DIR}/${DATE}"

mkdir -p "$BACKUP_DIR"

# Full backup with compression
mongodump --uri="$MONGO_URI" --gzip --out="$TARGET"

echo "[$(date)] MongoDB backup complete: $TARGET"

MongoDB 恢复

# Restore all databases
mongorestore --uri="mongodb://admin:secret@localhost:27017" \
  --gzip --drop /backups/mongodb/20250115_020000/

# Restore a single database
mongorestore --uri="mongodb://admin:secret@localhost:27017" \
  --gzip --drop --db mydb /backups/mongodb/20250115_020000/mydb/

# Restore a single collection
mongorestore --uri="mongodb://admin:secret@localhost:27017" \
  --gzip --drop --db mydb --collection users \
  /backups/mongodb/20250115_020000/mydb/users.bson.gz

上传到 S3

#!/bin/bash
# s3_upload.sh — Upload backups to S3
set -euo pipefail

S3_BUCKET="s3://my-backups"
BACKUP_DIR="/backups"
DATE=$(date +%Y%m%d)

# Upload PostgreSQL backup
aws s3 cp "${BACKUP_DIR}/postgres/" "${S3_BUCKET}/postgres/${DATE}/" \
  --recursive --storage-class STANDARD_IA \
  --sse AES256

# Upload MySQL backup
aws s3 cp "${BACKUP_DIR}/mysql/" "${S3_BUCKET}/mysql/${DATE}/" \
  --recursive --storage-class STANDARD_IA \
  --sse AES256

# Upload MongoDB backup
aws s3 cp "${BACKUP_DIR}/mongodb/" "${S3_BUCKET}/mongodb/${DATE}/" \
  --recursive --storage-class STANDARD_IA \
  --sse AES256

echo "[$(date)] S3 upload complete for ${DATE}"

S3 生命周期保留策略

{
  "Rules": [
    {
      "ID": "BackupRetention",
      "Status": "Enabled",
      "Filter": { "Prefix": "" },
      "Transitions": [
        { "Days": 30, "StorageClass": "GLACIER" }
      ],
      "Expiration": { "Days": 365 }
    }
  ]
}

aws s3api put-bucket-lifecycle-configuration \
  --bucket my-backups \
  --lifecycle-configuration file://lifecycle.json

Restic 备份（加密、重复数据删除）

# Initialize a restic repository on S3
export AWS_ACCESS_KEY_ID="AKIA..."
export AWS_SECRET_ACCESS_KEY="..."
export RESTIC_PASSWORD="strong_encryption_password"
export RESTIC_REPOSITORY="s3:s3.amazonaws.com/my-backups-restic"

restic init

# Backup the local backup directory
restic backup /backups/postgres /backups/mysql /backups/mongodb

# List snapshots
restic snapshots

# Prune old snapshots — keep 7 daily, 4 weekly, 6 monthly
restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prune

# Restore a snapshot
restic restore latest --target /restore/

Cron 计划

# /etc/cron.d/db-backups

# PostgreSQL: nightly at 02:00
0 2 * * * backup /opt/scripts/pg_backup.sh >> /var/log/backup-pg.log 2>&1

# MySQL: nightly at 02:30
30 2 * * * backup /opt/scripts/mysql_backup.sh >> /var/log/backup-mysql.log 2>&1

# MongoDB: nightly at 03:00
0 3 * * * backup /opt/scripts/mongo_backup.sh >> /var/log/backup-mongo.log 2>&1

# Upload to S3: daily at 04:00
0 4 * * * backup /opt/scripts/s3_upload.sh >> /var/log/backup-s3.log 2>&1

# Local cleanup: keep 7 days of local backups
0 5 * * * backup find /backups -type f -mtime +7 -delete >> /var/log/backup-cleanup.log 2>&1

# Restic prune: weekly on Sunday at 06:00
0 6 * * 0 backup /opt/scripts/restic_prune.sh >> /var/log/backup-restic.log 2>&1

自动化恢复测试

#!/bin/bash
# verify_backup.sh — weekly restore test
set -euo pipefail

BACKUP_FILE=$(ls -t /backups/postgres/mydb_*.dump | head -1)

echo "[$(date)] Starting backup verification with $BACKUP_FILE"

# Spin up a temporary PostgreSQL container
docker run -d --name pg-restore-test \
  -e POSTGRES_USER=testuser \
  -e POSTGRES_PASSWORD=testpass \
  -e POSTGRES_DB=testdb \
  postgres:16-alpine

# Wait for container to be ready
sleep 5
until docker exec pg-restore-test pg_isready -U testuser; do
  sleep 2
done

# Copy backup into container and restore
docker cp "$BACKUP_FILE" pg-restore-test:/tmp/backup.dump
docker exec pg-restore-test pg_restore -U testuser -d testdb --clean --if-exists /tmp/backup.dump

# Run verification queries
USERS_COUNT=$(docker exec pg-restore-test psql -U testuser -d testdb -tAc "SELECT COUNT(*) FROM users;")
ORDERS_COUNT=$(docker exec pg-restore-test psql -U testuser -d testdb -tAc "SELECT COUNT(*) FROM orders;")

echo "[$(date)] Verification: users=$USERS_COUNT, orders=$ORDERS_COUNT"

# Cleanup
docker rm -f pg-restore-test

# Alert on failure
if [ "$USERS_COUNT" -lt 1 ]; then
  echo "ALERT: Backup verification failed — users table is empty" >&2
  exit 1
fi

echo "[$(date)] Backup verification PASSED"

统一备份脚本

#!/bin/bash
# backup_all.sh — unified backup orchestrator
set -euo pipefail

LOG="/var/log/backup-all.log"
ALERT_EMAIL="[email protected]"
ERRORS=0

log() { echo "[$(date '+%F %T')] $*" | tee -a "$LOG"; }

run_backup() {
  local name="$1" script="$2"
  log "Starting $name backup..."
  if bash "$script" >> "$LOG" 2>&1; then
    log "$name backup succeeded."
  else
    log "ERROR: $name backup FAILED."
    ERRORS=$((ERRORS + 1))
  fi
}

run_backup "PostgreSQL" /opt/scripts/pg_backup.sh
run_backup "MySQL"      /opt/scripts/mysql_backup.sh
run_backup "MongoDB"    /opt/scripts/mongo_backup.sh
run_backup "S3 Upload"  /opt/scripts/s3_upload.sh

if [ "$ERRORS" -gt 0 ]; then
  log "Backup run completed with $ERRORS error(s). Sending alert."
  mail -s "BACKUP ALERT: $ERRORS failure(s)" "$ALERT_EMAIL" < "$LOG"
  exit 1
fi

log "All backups completed successfully."

最佳实践

• 3-2-1 规则：保留 3 份数据副本，存放在 2 种不同介质上，其中 1 份异地保存。
• 加密静态备份：使用 restic（内置加密）、AWS SSE 或 gpg。
• 定期测试还原：从未还原过的备份不算备份。
• 监控备份任务：出现任何故障时立即发出告警；不要依赖无提示的 cron 任务。
• 记录 RTO 和 RPO：为每个数据库定义恢复时间目标和恢复点目标。
• 对备份脚本进行版本控制：将脚本与基础设施代码一同存储在 Git 中。
• 使用 --single-transaction：为 MySQL 和 PostgreSQL 创建逻辑备份，以获取一致的快照。
• 分离备份凭据：为备份使用专用的只读数据库用户。

故障排除

症状	可能原因	解决方法
pg_dump: too many clients	备份连接与应用连接池争用连接	安排在低流量时段执行；为备份用户将 max_connections 增加 5
大表导致 mysqldump 挂起	表锁争用	使用 --single-transaction（InnoDB），或安排在维护窗口执行
副本上的 mongodump 速度缓慢	在高负载下从辅助节点读取	使用 --readPreference=secondaryPreferred，并安排在非高峰时段执行
S3 上传因超时失败	慢速连接上传大文件	使用 aws s3 cp --expected-size，或通过 aws s3api 进行分段上传
Restic prune 需要数小时	累积了过多快照	更频繁地运行 restic forget --prune；限制快照数量
恢复失败，提示 "role does not exist"	备份包含依赖角色的对象	先创建角色，或在恢复时使用 --no-owner --no-privileges

相关技能

• postgresql - PostgreSQL 管理和 pg_dump 详情
• mysql - MySQL 管理和 mysqldump 详情
• mongodb - MongoDB 管理和 mongodump 详情
• redis - Redis RDB/AOF 持久化和备份