mcpField
O
object-storage

object-storage

github.com/bagelhole云与 DevOps1.2k352 次安装23 周前更新
代码仓库

关于 object-storage

面向管理 AWS S3、MinIO 及兼容对象存储的技能，提供命令行操作和配置指导。涵盖存储桶访问策略、生命周期、版本控制与加密，也包括 MinIO 自托管和数据同步。

主要功能

  1. 01AWS S3 存储桶管理
  2. 02上传、下载与数据同步
  3. 03访问策略与 IAM 配置
  4. 04生命周期规则与版本控制
  5. 05MinIO 部署及 mc 管理

适用场景

  1. 01存储应用资产、备份、日志或媒体文件
  2. 02使用 MinIO 搭建本地 S3 兼容存储
  3. 03设置对象自动转换或过期规则
  4. 04配置存储桶访问控制

安装

复制下面的命令，在终端中运行。

$npx skills add https://github.com/bagelhole/devops-security-agent-skills --skill object-storage

安装后在 Claude Code 中输入/object-storage确认已加载。

详细介绍

对象存储

配置和管理对象存储解决方案，包括 AWS S3、MinIO（可自托管）和兼容的提供商。涵盖命令行操作、存储桶策略、生命周期规则、版本控制、加密和 MinIO 客户端（mc）。

适用场景

  • 存储应用资源、备份、日志或媒体文件
  • 使用 MinIO 在本地部署兼容 S3 的对象存储
  • 配置生命周期规则，自动转换或过期对象
  • 使用存储桶策略和 IAM 实施访问控制
  • 在本地文件系统与对象存储之间同步数据
  • 从 S3 或 MinIO 提供静态内容

前置条件

  • 已安装并配置 AWS CLI v2（S3 操作需运行 aws configure）
  • 已安装 Docker，以便自托管部署 MinIO
  • 已安装 MinIO 客户端（mc），用于管理 MinIO
  • 具有适当 S3 权限的 IAM 凭证
  • 可访问对象存储端点的网络连接

AWS S3 CLI 操作

存储桶管理
# Create a new bucket
aws s3 mb s3://my-app-assets-prod

# Create a bucket in a specific region
aws s3 mb s3://my-app-assets-eu --region eu-west-1

# List all buckets
aws s3 ls

# List objects in a bucket (with sizes)
aws s3 ls s3://my-app-assets-prod --recursive --human-readable --summarize

# Delete an empty bucket
aws s3 rb s3://my-old-bucket

# Delete a bucket and ALL its contents (destructive)
aws s3 rb s3://my-old-bucket --force
上传和下载
# Upload a single file
aws s3 cp ./report.pdf s3://my-app-assets-prod/reports/

# Upload with a specific storage class
aws s3 cp ./archive.tar.gz s3://my-app-assets-prod/archives/ --storage-class GLACIER

# Upload with server-side encryption (AES-256)
aws s3 cp ./sensitive.dat s3://my-app-assets-prod/data/ --sse AES256

# Download a file
aws s3 cp s3://my-app-assets-prod/reports/report.pdf ./downloads/

# Sync a local directory to S3 (upload only changed files)
aws s3 sync ./build/ s3://my-app-assets-prod/static/ --delete

# Sync from S3 to local
aws s3 sync s3://my-app-assets-prod/static/ ./local-copy/

# Sync with exclusion patterns
aws s3 sync ./logs/ s3://my-app-logs/ --exclude "*.tmp" --exclude ".git/*"

# Copy between buckets
aws s3 sync s3://source-bucket/ s3://destination-bucket/ --source-region us-east-1 --region eu-west-1

# Generate a pre-signed URL (temporary access, 1 hour)
aws s3 presign s3://my-app-assets-prod/reports/report.pdf --expires-in 3600

# Recursive delete of a prefix
aws s3 rm s3://my-app-assets-prod/old-data/ --recursive
版本控制
# Enable versioning on a bucket
aws s3api put-bucket-versioning \
  --bucket my-app-assets-prod \
  --versioning-configuration Status=Enabled

# Check versioning status
aws s3api get-bucket-versioning --bucket my-app-assets-prod

# List object versions
aws s3api list-object-versions --bucket my-app-assets-prod --prefix reports/

# Restore a previous version (copy old version to current)
aws s3api copy-object \
  --bucket my-app-assets-prod \
  --copy-source "my-app-assets-prod/reports/report.pdf?versionId=abc123" \
  --key reports/report.pdf

# Delete a specific version permanently
aws s3api delete-object \
  --bucket my-app-assets-prod \
  --key reports/old-report.pdf \
  --version-id abc123
存储桶策略
# Apply a bucket policy from a JSON file
aws s3api put-bucket-policy --bucket my-app-assets-prod --policy file://policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "PublicReadForStaticSite",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::my-app-assets-prod/static/*"
    },
    {
      "Sid": "DenyUnencryptedUploads",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::my-app-assets-prod/*",
      "Condition": {
        "StringNotEquals": {
          "s3:x-amz-server-side-encryption": "AES256"
        }
      }
    },
    {
      "Sid": "RestrictToVPC",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::my-app-assets-prod",
        "arn:aws:s3:::my-app-assets-prod/*"
      ],
      "Condition": {
        "StringNotEquals": {
          "aws:sourceVpce": "vpce-abc123"
        }
      }
    }
  ]
}
生命周期规则
# Apply lifecycle configuration
aws s3api put-bucket-lifecycle-configuration \
  --bucket my-app-assets-prod \
  --lifecycle-configuration file://lifecycle.json

{
  "Rules": [
    {
      "ID": "TransitionLogsToIA",
      "Filter": { "Prefix": "logs/" },
      "Status": "Enabled",
      "Transitions": [
        {
          "Days": 30,
          "StorageClass": "STANDARD_IA"
        },
        {
          "Days": 90,
          "StorageClass": "GLACIER"
        }
      ],
      "Expiration": {
        "Days": 365
      }
    },
    {
      "ID": "CleanupIncompleteUploads",
      "Filter": { "Prefix": "" },
      "Status": "Enabled",
      "AbortIncompleteMultipartUpload": {
        "DaysAfterInitiation": 7
      }
    },
    {
      "ID": "ExpireOldVersions",
      "Filter": { "Prefix": "" },
      "Status": "Enabled",
      "NoncurrentVersionExpiration": {
        "NoncurrentDays": 30
      }
    }
  ]
}

# View current lifecycle rules
aws s3api get-bucket-lifecycle-configuration --bucket my-app-assets-prod

# Enable S3 access logging
aws s3api put-bucket-logging --bucket my-app-assets-prod --bucket-logging-status '{
  "LoggingEnabled": {
    "TargetBucket": "my-app-logs",
    "TargetPrefix": "s3-access-logs/"
  }
}'

MinIO 自托管部署

Docker 部署
# Single-node MinIO with persistent storage
docker run -d \
  --name minio \
  --restart unless-stopped \
  -p 9000:9000 \
  -p 9001:9001 \
  -e MINIO_ROOT_USER=minioadmin \
  -e MINIO_ROOT_PASSWORD=minio-secret-key-change-me \
  -v /data/minio:/data \
  minio/minio server /data --console-address ":9001"
Docker Compose（多驱动器）
# docker-compose.yml
version: "3.8"
services:
  minio:
    image: minio/minio:latest
    command: server /data{1...4} --console-address ":9001"
    environment:
      MINIO_ROOT_USER: minioadmin
      MINIO_ROOT_PASSWORD: minio-secret-key-change-me
      MINIO_BROWSER_REDIRECT_URL: https://minio-console.example.com
    ports:
      - "9000:9000"
      - "9001:9001"
    volumes:
      - minio-data1:/data1
      - minio-data2:/data2
      - minio-data3:/data3
      - minio-data4:/data4
    healthcheck:
      test: ["CMD", "mc", "ready", "local"]
      interval: 30s
      timeout: 10s
      retries: 3
    restart: unless-stopped

volumes:
  minio-data1:
  minio-data2:
  minio-data3:
  minio-data4:

# Start the stack
docker compose up -d

# Check health
docker compose ps
curl -s http://localhost:9000/minio/health/live
MinIO 客户端（mc）命令
# Install mc
curl -O https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc && mv mc /usr/local/bin/

# Configure an alias for the MinIO server
mc alias set myminio http://localhost:9000 minioadmin minio-secret-key-change-me

# Configure an alias for AWS S3
mc alias set aws https://s3.amazonaws.com AKIAEXAMPLE SECRETKEYEXAMPLE

# Bucket operations
mc mb myminio/app-data
mc mb myminio/backups
mc ls myminio/

# Upload and download
mc cp ./backup.tar.gz myminio/backups/
mc cp myminio/backups/backup.tar.gz ./restore/

# Sync a directory (mirror)
mc mirror ./static/ myminio/app-data/static/
mc mirror --watch ./static/ myminio/app-data/static/   # Continuous sync

# Set bucket policy (download = public read)
mc anonymous set download myminio/app-data/static

# Set a specific policy from JSON
mc anonymous set-json policy.json myminio/app-data

# Enable versioning
mc version enable myminio/app-data

# Set lifecycle rule: expire objects in tmp/ after 7 days
mc ilm rule add --expiry-days 7 --prefix "tmp/" myminio/app-data

# List lifecycle rules
mc ilm rule ls myminio/app-data

# Create a service account (for applications)
mc admin user svcacct add myminio minioadmin --access-key myapp-key --secret-key myapp-secret

# View server info and disk usage
mc admin info myminio

# Check bucket disk usage
mc du myminio/app-data

# Set a notification target (webhook on object creation)
mc event add myminio/app-data arn:minio:sqs::myqueue:webhook --event put
mc event ls myminio/app-data

故障排除

症状诊断命令常见修复方法
S3 访问被拒绝aws s3api get-bucket-policy --bucket name检查 IAM 策略、存储桶策略和阻止公开访问设置
上传缓慢aws s3 cp --debug使用分段上传：aws configure set s3.multipart_threshold 64MB
预签名 URL 返回 403检查时钟偏差和 URL 过期时间使用 NTP 同步系统时钟；重新生成 URL
MinIO 状态不正常mc admin info myminio检查磁盘空间、容器日志和端口可用性
生命周期规则未生效aws s3api get-bucket-lifecycle-configuration规则每天运行一次；检查 Filter 前缀是否匹配
对象未启用版本控制aws s3api get-bucket-versioning启用版本控制；此操作不会追溯生效
mc：连接被拒绝mc alias ls验证端点 URL、端口和凭证
大规模同步缓慢使用 mc mirror --watch 进行监控使用 --multi-thread 标志并提高带宽

相关技能

  • block-storage -- MinIO 数据卷的底层磁盘存储
  • backup-recovery -- 使用 restic 将 S3/MinIO 用作备份目标
  • nfs-storage -- 用于文件级访问的共享存储替代方案
  • linux-administration -- MinIO 主机的服务器设置和维护

评论

支持 Markdown