aws-s3

Name: aws-s3
Author: bagelhole

github.com/bagelhole云与 DevOps1.2k347 次安装23 周前更新

代码仓库

关于 aws-s3

为使用 AWS CLI v2 管理 Amazon S3 的人员提供操作指导，涵盖存储桶安全、生命周期、跨区域复制和访问控制，并包含常见故障排查。

主要功能

01创建并强化存储桶安全
02配置存储桶策略与访问控制
03设置对象生命周期规则
04配置跨区域复制
05生成预签名 URL
06排查权限与策略问题

适用场景

01为生产数据创建安全存储桶
02设置对象向低成本存储类别迁移
03配置跨区域灾难恢复
04为私有对象提供临时访问

安装

复制下面的命令，在终端中运行。

$npx skills add https://github.com/bagelhole/devops-security-agent-skills --skill aws-s3

安装后在 Claude Code 中输入/aws-s3确认已加载。

详细介绍

AWS S3

通过生产级安全性、生命周期策略、复制和访问控制来管理 Amazon S3 对象存储。

何时使用此技能

创建具有安全加固配置（加密、阻止公有访问、版本控制）的 S3 存储桶
编写存储桶策略，以强制使用 HTTPS、限制 IP 范围或授予跨账户访问权限
设置生命周期规则，在不同存储类别之间转换对象
配置跨区域复制以进行灾难恢复
生成预签名 URL，以便临时访问私有对象
设置静态网站托管或 CloudFront 源站
排查访问被拒绝错误或策略冲突

前置条件

已安装并配置 AWS CLI v2
IAM 权限：s3:*、s3-object-lambda:*、kms:*（用于 SSE-KMS）
进行复制时：具有复制权限的 IAM 角色，以及位于目标区域的目标存储桶
进行日志记录时：具有适当 ACL 的独立日志存储桶

创建并保护存储桶

# Create a bucket (us-east-1 does not need LocationConstraint)
aws s3api create-bucket \
  --bucket my-app-data-prod \
  --region us-east-1

# Create a bucket in another region
aws s3api create-bucket \
  --bucket my-app-data-dr \
  --region us-west-2 \
  --create-bucket-configuration LocationConstraint=us-west-2

# Block ALL public access (always do this first)
aws s3api put-public-access-block \
  --bucket my-app-data-prod \
  --public-access-block-configuration '{
    "BlockPublicAcls": true,
    "IgnorePublicAcls": true,
    "BlockPublicPolicy": true,
    "RestrictPublicBuckets": true
  }'

# Enable versioning
aws s3api put-bucket-versioning \
  --bucket my-app-data-prod \
  --versioning-configuration Status=Enabled

# Enable server-side encryption with SSE-KMS
aws s3api put-bucket-encryption \
  --bucket my-app-data-prod \
  --server-side-encryption-configuration '{
    "Rules": [{
      "ApplyServerSideEncryptionByDefault": {
        "SSEAlgorithm": "aws:kms",
        "KMSMasterKeyID": "alias/s3-key"
      },
      "BucketKeyEnabled": true
    }]
  }'

# Enable access logging
aws s3api put-bucket-logging \
  --bucket my-app-data-prod \
  --bucket-logging-status '{
    "LoggingEnabled": {
      "TargetBucket": "my-access-logs-bucket",
      "TargetPrefix": "s3-logs/my-app-data-prod/"
    }
  }'

# Add tags
aws s3api put-bucket-tagging \
  --bucket my-app-data-prod \
  --tagging '{
    "TagSet": [
      {"Key": "Environment", "Value": "production"},
      {"Key": "Team", "Value": "platform"},
      {"Key": "DataClassification", "Value": "confidential"}
    ]
  }'

存储桶策略

# Apply a bucket policy (enforce HTTPS and restrict to VPC endpoint)
aws s3api put-bucket-policy \
  --bucket my-app-data-prod \
  --policy '{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Sid": "DenyInsecureTransport",
        "Effect": "Deny",
        "Principal": "*",
        "Action": "s3:*",
        "Resource": [
          "arn:aws:s3:::my-app-data-prod",
          "arn:aws:s3:::my-app-data-prod/*"
        ],
        "Condition": {
          "Bool": {"aws:SecureTransport": "false"}
        }
      },
      {
        "Sid": "RestrictToVPCEndpoint",
        "Effect": "Deny",
        "Principal": "*",
        "Action": "s3:*",
        "Resource": [
          "arn:aws:s3:::my-app-data-prod",
          "arn:aws:s3:::my-app-data-prod/*"
        ],
        "Condition": {
          "StringNotEquals": {
            "aws:sourceVpce": "vpce-abc123"
          }
        }
      }
    ]
  }'

跨账户访问策略：

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "CrossAccountRead",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::987654321098:role/DataAnalystRole"
      },
      "Action": [
        "s3:GetObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::my-app-data-prod",
        "arn:aws:s3:::my-app-data-prod/shared/*"
      ]
    }
  ]
}

生命周期规则

# Apply a comprehensive lifecycle configuration
aws s3api put-bucket-lifecycle-configuration \
  --bucket my-app-data-prod \
  --lifecycle-configuration '{
    "Rules": [
      {
        "ID": "TierDownOldData",
        "Status": "Enabled",
        "Filter": {"Prefix": "data/"},
        "Transitions": [
          {"Days": 30, "StorageClass": "STANDARD_IA"},
          {"Days": 90, "StorageClass": "GLACIER_IR"},
          {"Days": 180, "StorageClass": "GLACIER"},
          {"Days": 365, "StorageClass": "DEEP_ARCHIVE"}
        ]
      },
      {
        "ID": "ExpireLogs",
        "Status": "Enabled",
        "Filter": {"Prefix": "logs/"},
        "Expiration": {"Days": 90},
        "Transitions": [
          {"Days": 7, "StorageClass": "STANDARD_IA"},
          {"Days": 30, "StorageClass": "GLACIER"}
        ]
      },
      {
        "ID": "CleanupOldVersions",
        "Status": "Enabled",
        "Filter": {"Prefix": ""},
        "NoncurrentVersionTransitions": [
          {"NoncurrentDays": 30, "StorageClass": "STANDARD_IA"},
          {"NoncurrentDays": 90, "StorageClass": "GLACIER"}
        ],
        "NoncurrentVersionExpiration": {"NoncurrentDays": 180}
      },
      {
        "ID": "AbortIncompleteUploads",
        "Status": "Enabled",
        "Filter": {"Prefix": ""},
        "AbortIncompleteMultipartUpload": {"DaysAfterInitiation": 7}
      },
      {
        "ID": "ExpireDeleteMarkers",
        "Status": "Enabled",
        "Filter": {"Prefix": ""},
        "Expiration": {"ExpiredObjectDeleteMarker": true}
      }
    ]
  }'

跨区域复制

# Enable replication (requires versioning on both buckets)
aws s3api put-bucket-replication \
  --bucket my-app-data-prod \
  --replication-configuration '{
    "Role": "arn:aws:iam::123456789012:role/S3ReplicationRole",
    "Rules": [
      {
        "ID": "ReplicateAll",
        "Status": "Enabled",
        "Priority": 1,
        "Filter": {"Prefix": ""},
        "Destination": {
          "Bucket": "arn:aws:s3:::my-app-data-dr",
          "StorageClass": "STANDARD_IA",
          "EncryptionConfiguration": {
            "ReplicaKmsKeyID": "arn:aws:kms:us-west-2:123456789012:key/dr-key-id"
          },
          "Metrics": {"Status": "Enabled", "EventThreshold": {"Minutes": 15}},
          "ReplicationTime": {"Status": "Enabled", "Time": {"Minutes": 15}}
        },
        "DeleteMarkerReplication": {"Status": "Enabled"},
        "SourceSelectionCriteria": {
          "SseKmsEncryptedObjects": {"Status": "Enabled"}
        }
      }
    ]
  }'

# Check replication status
aws s3api head-object \
  --bucket my-app-data-prod \
  --key data/important-file.json \
  --query "ReplicationStatus"

预签名 URL

# Generate a presigned URL for downloading (valid 1 hour)
aws s3 presign s3://my-app-data-prod/reports/quarterly.pdf \
  --expires-in 3600

# Generate a presigned URL for uploading
aws s3 presign s3://my-app-data-prod/uploads/user-file.zip \
  --expires-in 3600

# Presigned URL with specific content type (using the API directly)
aws s3api generate-presigned-url \
  --client-method put_object \
  --params '{"Bucket":"my-app-data-prod","Key":"uploads/photo.jpg","ContentType":"image/jpeg"}' \
  --expires-in 3600

常见 S3 操作

# Sync a local directory to S3
aws s3 sync ./build s3://my-app-data-prod/static/ \
  --delete \
  --exclude "*.tmp" \
  --cache-control "max-age=31536000" \
  --content-encoding "gzip"

# Copy with storage class
aws s3 cp large-archive.tar.gz s3://my-app-data-prod/archives/ \
  --storage-class GLACIER_IR

# List objects with size summary
aws s3 ls s3://my-app-data-prod/ --recursive --summarize --human-readable

# Remove all objects with a prefix
aws s3 rm s3://my-app-data-prod/temp/ --recursive

# Get bucket size via CloudWatch (most efficient for large buckets)
aws cloudwatch get-metric-statistics \
  --namespace AWS/S3 \
  --metric-name BucketSizeBytes \
  --dimensions Name=BucketName,Value=my-app-data-prod Name=StorageType,Value=StandardStorage \
  --start-time "$(date -u -d '2 days ago' +%Y-%m-%dT%H:%M:%SZ)" \
  --end-time "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  --period 86400 \
  --statistics Average \
  --output table

Terraform S3 存储桶

resource "aws_s3_bucket" "main" {
  bucket = "my-app-data-prod"

  tags = {
    Environment        = "production"
    DataClassification = "confidential"
  }
}

resource "aws_s3_bucket_versioning" "main" {
  bucket = aws_s3_bucket.main.id
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_public_access_block" "main" {
  bucket = aws_s3_bucket.main.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

resource "aws_s3_bucket_server_side_encryption_configuration" "main" {
  bucket = aws_s3_bucket.main.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm     = "aws:kms"
      kms_master_key_id = aws_kms_key.s3.arn
    }
    bucket_key_enabled = true
  }
}

resource "aws_s3_bucket_lifecycle_configuration" "main" {
  bucket = aws_s3_bucket.main.id

  rule {
    id     = "tier-down"
    status = "Enabled"

    transition {
      days          = 30
      storage_class = "STANDARD_IA"
    }

    transition {
      days          = 90
      storage_class = "GLACIER"
    }

    noncurrent_version_transition {
      noncurrent_days = 30
      storage_class   = "GLACIER"
    }

    noncurrent_version_expiration {
      noncurrent_days = 180
    }

    abort_incomplete_multipart_upload {
      days_after_initiation = 7
    }
  }
}

resource "aws_s3_bucket_policy" "enforce_https" {
  bucket = aws_s3_bucket.main.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Sid       = "DenyInsecureTransport"
      Effect    = "Deny"
      Principal = "*"
      Action    = "s3:*"
      Resource  = [
        aws_s3_bucket.main.arn,
        "${aws_s3_bucket.main.arn}/*"
      ]
      Condition = {
        Bool = { "aws:SecureTransport" = "false" }
      }
    }]
  })
}

故障排查

问题	原因	修复方法
GetObject 返回 Access Denied	存储桶策略或 IAM 拒绝访问	检查存储桶策略、IAM 策略和公有访问阻止设置
PutObject 返回 Access Denied	必需的加密标头缺失	添加 SSE 标头；检查存储桶策略中的加密条件
预签名 URL 返回 403	URL 已过期或区域错误	重新生成；确保区域与存储桶所在区域一致
复制未生效	源存储桶或目标存储桶未启用版本控制	在两个存储桶上启用版本控制
生命周期转换未生效	规则筛选条件与对象不匹配	验证前缀和标签筛选条件；检查规则状态
删除存储桶失败	存储桶非空或包含启用了版本控制的对象	先删除所有对象和版本；禁用版本控制
大文件上传缓慢	使用了单段上传	使用 `aws s3 cp`（自动分段上传）或设置分段阈值
跨账户访问被拒绝	需要同时配置存储桶策略和 IAM 策略	在存储桶策略和调用方的 IAM 策略中授予权限
对象锁定阻止删除	治理模式或合规模式已启用	使用治理模式绕过（需具备相应权限）或等待保留期结束

基本信息

作者: bagelhole
仓库: bagelhole/devops-security-agent-skills
分类: 云与 DevOps
安装次数: 347
GitHub Stars: 1,180

报告问题