implementing-aws-macie-for-data-classification

实施 AWS Macie 进行数据分类

概述

Amazon Macie 是一项完全托管的数据安全与隐私服务，利用机器学习和模式匹配来发现并保护 Amazon S3 中的敏感数据。Macie 每天自动评估你的 S3 存储桶清单，并识别包含 PII、财务信息、凭证及其他敏感数据类型的对象。它提供两种发现方式：用于广泛可见性的自动化敏感数据发现，以及用于深度分析的目标发现作业。

何时使用

• 在环境中部署或配置实施 AWS Macie 进行数据分类能力时
• 在建立符合合规要求的安全控制措施时
• 在为该领域构建或改进安全架构时
• 在进行需要此实施的安全评估时

前提条件

• 拥有包含待分类数据的 S3 存储桶的 AWS 账户
• 用于 Macie 服务配置的 IAM 权限
• AWS Organizations 设置（用于多账户部署）
• 位于受支持区域的 S3 存储桶

启用 Macie

通过 AWS CLI

# Enable Macie in the current account/region
aws macie2 enable-macie

# Verify Macie is enabled
aws macie2 get-macie-session

# Enable automated sensitive data discovery
aws macie2 update-automated-discovery-configuration \
  --status ENABLED

通过 Terraform

resource "aws_macie2_account" "main" {}

resource "aws_macie2_classification_export_configuration" "main" {
  depends_on = [aws_macie2_account.main]

  s3_destination {
    bucket_name = aws_s3_bucket.macie_results.id
    key_prefix  = "macie-findings/"
    kms_key_arn = aws_kms_key.macie.arn
  }
}

配置发现作业

为特定存储桶创建分类作业

aws macie2 create-classification-job \
  --job-type ONE_TIME \
  --name "pii-scan-production-buckets" \
  --s3-job-definition '{
    "bucketDefinitions": [{
      "accountId": "123456789012",
      "buckets": [
        "production-data-bucket",
        "customer-records-bucket"
      ]
    }]
  }' \
  --managed-data-identifier-selector ALL

创建定时循环作业

aws macie2 create-classification-job \
  --job-type SCHEDULED \
  --name "weekly-sensitive-data-scan" \
  --schedule-frequency-details '{
    "weekly": {
      "dayOfWeek": "MONDAY"
    }
  }' \
  --s3-job-definition '{
    "bucketDefinitions": [{
      "accountId": "123456789012",
      "buckets": ["all-data-bucket"]
    }],
    "scoping": {
      "includes": {
        "and": [{
          "simpleScopeTerm": {
            "comparator": "STARTS_WITH",
            "key": "OBJECT_KEY",
            "values": ["uploads/", "documents/"]
          }
        }]
      }
    }
  }'

自定义数据标识符

为内部 ID 创建自定义标识符

aws macie2 create-custom-data-identifier \
  --name "internal-employee-id" \
  --description "Matches internal employee ID format EMP-XXXXXX" \
  --regex "EMP-[0-9]{6}" \
  --severity-levels '[
    {"occurrencesThreshold": 1, "severity": "LOW"},
    {"occurrencesThreshold": 10, "severity": "MEDIUM"},
    {"occurrencesThreshold": 50, "severity": "HIGH"}
  ]'

为项目代码创建标识符

aws macie2 create-custom-data-identifier \
  --name "project-code-identifier" \
  --description "Matches project codes in format PRJ-XXXX-XX" \
  --regex "PRJ-[A-Z]{4}-[0-9]{2}" \
  --keywords '["project", "code", "initiative"]' \
  --maximum-match-distance 50

允许列表

创建允许列表以抑制误报

aws macie2 create-allow-list \
  --name "test-data-exclusions" \
  --description "Exclude known test data patterns" \
  --criteria '{
    "regex": "TEST-[0-9]{4}-[0-9]{4}-[0-9]{4}-[0-9]{4}"
  }'

托管数据标识符

Macie 提供 300+ 托管数据标识符，涵盖：

类别	示例
PII	SSN、护照号码、驾照、出生日期、姓名、地址
财务	信用卡号、银行账号、SWIFT 代码
凭证	AWS 密钥、API 密钥、SSH 私钥、OAuth 令牌
健康	HIPAA 标识符、健康保险索赔号码
法律	税务识别号、国民身份证号

发现结果管理

列出发现结果

# Get sensitive data findings
aws macie2 list-findings \
  --finding-criteria '{
    "criterion": {
      "severity.description": {
        "eq": ["High"]
      },
      "category": {
        "eq": ["CLASSIFICATION"]
      }
    }
  }' \
  --sort-criteria '{"attributeName": "updatedAt", "orderBy": "DESC"}' \
  --max-results 25

获取发现结果详情

aws macie2 get-findings \
  --finding-ids '["finding-id-1", "finding-id-2"]'

将发现结果导出到 Security Hub

# Macie automatically publishes findings to Security Hub
# Verify integration:
aws macie2 get-macie-session --query 'findingPublishingFrequency'

用于自动化响应的 EventBridge 集成

{
  "source": ["aws.macie"],
  "detail-type": ["Macie Finding"],
  "detail": {
    "severity": {
      "description": ["High", "Critical"]
    }
  }
}

用于自动化修复的 Lambda 函数

import boto3
import json

s3 = boto3.client('s3')
sns = boto3.client('sns')

def lambda_handler(event, context):
    finding = event['detail']
    severity = finding['severity']['description']
    bucket = finding['resourcesAffected']['s3Bucket']['name']
    key = finding['resourcesAffected']['s3Object']['key']
    sensitive_types = [d['type'] for d in finding.get('classificationDetails', {}).get('result', {}).get('sensitiveData', [])]

    if severity in ['High', 'Critical']:
        # Tag the object for review
        s3.put_object_tagging(
            Bucket=bucket,
            Key=key,
            Tagging={
                'TagSet': [
                    {'Key': 'macie-finding', 'Value': severity},
                    {'Key': 'sensitive-data', 'Value': ','.join(sensitive_types)},
                    {'Key': 'requires-review', 'Value': 'true'}
                ]
            }
        )

        # Notify security team
        sns.publish(
            TopicArn='arn:aws:sns:us-east-1:123456789012:security-alerts',
            Subject=f'Macie {severity} Finding: {bucket}/{key}',
            Message=json.dumps({
                'bucket': bucket,
                'key': key,
                'severity': severity,
                'sensitive_data_types': sensitive_types,
                'finding_id': finding['id']
            }, indent=2)
        )

    return {'statusCode': 200}

多账户部署

指定 Macie 管理员账户

# From the management account
aws macie2 enable-organization-admin-account \
  --admin-account-id 111111111111

添加成员账户

# From the administrator account
aws macie2 create-member \
  --account '{"accountId": "222222222222", "email": "[email protected]"}'

监控 Macie 操作

使用统计

aws macie2 get-usage-statistics \
  --filter-by '[{"comparator": "GT", "key": "accountId", "values": []}]' \
  --sort-by '{"key": "accountId", "orderBy": "ASC"}'

分类作业状态

aws macie2 list-classification-jobs \
  --filter-criteria '{"includes": [{"comparator": "EQ", "key": "jobStatus", "values": ["RUNNING"]}]}'

参考

• AWS Macie 文档：https://docs.aws.amazon.com/macie/
• AWS Macie 定价
• Macie 分析支持的文件类型
• 使用 Macie 实现 GDPR 和 CCPA 合规