exploiting-excessive-data-exposure-in-api

利用 API 中的过度数据暴露

何时使用

• 测试前端只显示部分数据但 API 响应中包含额外字段的 API
• 评估移动应用 API，这类响应是为多种客户端类型设计的，可能包含多余数据
• 识别 API 响应中的 PII 泄露，包括 UI 中未显示的电子邮件地址、电话号码、SSN 或支付数据
• 测试 GraphQL API，其中客户端可以请求包括敏感属性在内的任意字段
• 在微服务重构后评估 API，此时内部服务间数据泄露到公共端点

请勿使用，除非获得书面授权。数据暴露测试涉及捕获和分析可能敏感的个人数据。

前提条件

• 明确指定目标 API 端点和范围的书面授权
• 配置为拦截代理的 Burp Suite Professional 或 mitmproxy
• 两个不同权限级别的测试账户（普通用户和管理员）
• 用于流量捕获的浏览器开发者工具或移动端代理设置
• Python 3.10+ 以及 requests 和 json 库
• API 文档（OpenAPI 规范）用于与实际响应进行对比

法律声明： 此技能仅用于经授权的安全测试和教育目的。未经授权对您不拥有或未获得书面许可测试的系统使用是违法的，并可能违反计算机欺诈法律。

工作流

步骤 1：响应模式发现

将文档化的 API 响应与实际响应进行对比：

import requests
import json

BASE_URL = "https://target-api.example.com/api/v1"
headers = {"Authorization": "Bearer <user_token>", "Content-Type": "application/json"}

# Fetch a resource and analyze all returned fields
endpoints_to_test = [
    ("GET", "/users/me", None),
    ("GET", "/users/me/orders", None),
    ("GET", "/products", None),
    ("GET", "/users/me/settings", None),
    ("GET", "/transactions", None),
]

for method, path, body in endpoints_to_test:
    resp = requests.request(method, f"{BASE_URL}{path}", headers=headers, json=body)
    if resp.status_code == 200:
        data = resp.json()
        # Recursively extract all field names
        def extract_fields(obj, prefix=""):
            fields = []
            if isinstance(obj, dict):
                for k, v in obj.items():
                    full_key = f"{prefix}.{k}" if prefix else k
                    fields.append(full_key)
                    fields.extend(extract_fields(v, full_key))
            elif isinstance(obj, list) and obj:
                fields.extend(extract_fields(obj[0], f"{prefix}[]"))
            return fields

        all_fields = extract_fields(data)
        print(f"\n{method} {path} - {len(all_fields)} fields returned:")
        for f in sorted(all_fields):
            print(f"  {f}")

步骤 2：敏感数据模式检测

扫描 API 响应中的敏感数据模式：

import re

SENSITIVE_PATTERNS = {
    "email": r'[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}',
    "phone": r'(\+?1?\s?\(?\d{3}\)?[\s.-]?\d{3}[\s.-]?\d{4})',
    "ssn": r'\b\d{3}-\d{2}-\d{4}\b',
    "credit_card": r'\b(?:4[0-9]{12}(?:[0-9]{3})?|5[1-5][0-9]{14}|3[47][0-9]{13})\b',
    "password_hash": r'\$2[aby]?\$\d{2}\$[./A-Za-z0-9]{53}',
    "api_key": r'(?:api[_-]?key|apikey)["\s:=]+["\']?([a-zA-Z0-9_\-]{20,})',
    "internal_ip": r'\b(?:10\.\d{1,3}|172\.(?:1[6-9]|2\d|3[01])|192\.168)\.\d{1,3}\.\d{1,3}\b',
    "aws_key": r'AKIA[0-9A-Z]{16}',
    "jwt_token": r'eyJ[A-Za-z0-9_-]+\.eyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+',
    "uuid": r'[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}',
}

SENSITIVE_FIELD_NAMES = [
    "password", "password_hash", "secret", "token", "ssn", "social_security",
    "credit_card", "card_number", "cvv", "pin", "private_key", "api_key",
    "internal_id", "debug", "trace", "stack_trace", "created_by_ip",
    "last_login_ip", "salt", "session_id", "refresh_token", "mfa_secret",
    "date_of_birth", "bank_account", "routing_number", "tax_id"
]

def scan_response(endpoint, response_text):
    findings = []
    # Check for sensitive data patterns in values
    for pattern_name, pattern in SENSITIVE_PATTERNS.items():
        matches = re.findall(pattern, response_text)
        if matches:
            findings.append({
                "endpoint": endpoint,
                "type": "sensitive_value",
                "pattern": pattern_name,
                "count": len(matches),
                "sample": matches[0][:20] + "..." if len(matches[0]) > 20 else matches[0]
            })

    # Check for sensitive field names
    response_lower = response_text.lower()
    for field in SENSITIVE_FIELD_NAMES:
        if f'"{field}"' in response_lower or f"'{field}'" in response_lower:
            findings.append({
                "endpoint": endpoint,
                "type": "sensitive_field",
                "field_name": field
            })

    return findings

# Scan all endpoint responses
for method, path, body in endpoints_to_test:
    resp = requests.request(method, f"{BASE_URL}{path}", headers=headers, json=body)
    if resp.status_code == 200:
        findings = scan_response(f"{method} {path}", resp.text)
        for f in findings:
            print(f"[FINDING] {f['endpoint']}: {f['type']} - {f.get('pattern', f.get('field_name'))}")

步骤 3：对比 UI 显示与 API 响应

# Fields the UI shows (observed from the frontend application)
ui_displayed_fields = {
    "/users/me": {"name", "email", "avatar_url", "role"},
    "/users/me/orders": {"order_id", "date", "status", "total"},
    "/products": {"id", "name", "price", "image_url", "description"},
}

# Fields the API actually returns
for method, path, body in endpoints_to_test:
    resp = requests.request(method, f"{BASE_URL}{path}", headers=headers, json=body)
    if resp.status_code == 200:
        data = resp.json()
        if isinstance(data, list):
            actual_fields = set(data[0].keys()) if data else set()
        elif isinstance(data, dict):
            # Handle paginated responses
            items_key = next((k for k in data if isinstance(data[k], list)), None)
            if items_key and data[items_key]:
                actual_fields = set(data[items_key][0].keys())
            else:
                actual_fields = set(data.keys())
        else:
            continue

        expected = ui_displayed_fields.get(path, set())
        excess = actual_fields - expected
        if excess:
            print(f"\n{method} {path} - EXCESS FIELDS (not shown in UI):")
            for field in sorted(excess):
                print(f"  - {field}")

步骤 4：测试相关端点中的用户对象暴露

# Many APIs embed full user objects in responses for orders, comments, etc.
endpoints_with_user_objects = [
    "/orders",          # Each order may include full seller/buyer profile
    "/comments",        # Comments may include full author profile
    "/reviews",         # Reviews may expose reviewer details
    "/transactions",    # Transactions may include counterparty info
    "/team/members",    # Team listing may expose excessive member data
]

for path in endpoints_with_user_objects:
    resp = requests.get(f"{BASE_URL}{path}", headers=headers)
    if resp.status_code == 200:
        text = resp.text
        # Check for user data leakage in nested objects
        user_fields_found = []
        for field in ["password_hash", "last_login_ip", "mfa_enabled", "phone_number",
                      "date_of_birth", "ssn", "internal_notes", "salary", "address"]:
            if f'"{field}"' in text:
                user_fields_found.append(field)
        if user_fields_found:
            print(f"[EXCESSIVE] {path} exposes user fields: {user_fields_found}")

步骤 5：GraphQL 过度获取分析

# GraphQL allows clients to request any available field
GRAPHQL_URL = f"{BASE_URL}/graphql"

# Introspection query to discover all fields on User type
introspection = {
    "query": """
    {
      __type(name: "User") {
        fields {
          name
          type {
            name
            kind
          }
        }
      }
    }
    """
}

resp = requests.post(GRAPHQL_URL, headers=headers, json=introspection)
if resp.status_code == 200:
    fields = resp.json().get("data", {}).get("__type", {}).get("fields", [])
    print("Available User fields via GraphQL:")
    for f in fields:
        sensitivity = "SENSITIVE" if f["name"] in SENSITIVE_FIELD_NAMES else "normal"
        print(f"  {f['name']} ({f['type']['name']}) [{sensitivity}]")

# Try to query sensitive fields
sensitive_query = {
    "query": """
    query {
      users {
        id
        email
        passwordHash
        socialSecurityNumber
        internalNotes
        lastLoginIp
        mfaSecret
        apiKey
      }
    }
    """
}
resp = requests.post(GRAPHQL_URL, headers=headers, json=sensitive_query)
if resp.status_code == 200 and "errors" not in resp.json():
    print("[CRITICAL] GraphQL exposes sensitive user fields without restriction")

步骤 6：调试和内部数据泄露

# Test for debug information in responses
debug_headers_to_check = [
    "X-Debug-Token", "X-Debug-Info", "Server", "X-Powered-By",
    "X-Request-Id", "X-Correlation-Id", "X-Backend-Server",
    "X-Runtime", "X-Version", "X-Build-Version"
]

resp = requests.get(f"{BASE_URL}/users/me", headers=headers)
for h in debug_headers_to_check:
    if h.lower() in {k.lower(): v for k, v in resp.headers.items()}:
        print(f"[INFO LEAK] Header {h}: {resp.headers.get(h)}")

# Test error responses for stack traces
error_payloads = [
    ("GET", "/users/invalid-id-format", None),
    ("POST", "/orders", {"invalid": "payload"}),
    ("GET", "/users/-1", None),
    ("GET", "/users/0", None),
]

for method, path, body in error_payloads:
    resp = requests.request(method, f"{BASE_URL}{path}", headers=headers, json=body)
    if resp.status_code >= 400:
        text = resp.text.lower()
        if any(kw in text for kw in ["stack trace", "traceback", "at com.", "at org.",
                                      "file \"", "line ", "exception", "sql", "query"]):
            print(f"[DEBUG LEAK] {method} {path} -> {resp.status_code}: Contains stack trace or query info")

关键概念

术语	定义
过度数据暴露	API 返回的数据字段超出客户端所需，依赖前端过滤来向用户隐藏敏感信息
过度获取	针对特定操作请求或接收超出所需的数据，常见于返回固定响应模式的 REST API
响应过滤	对 API 响应数据进行客户端过滤以仅显示相关字段，这提供零安全性，因为完整响应可被拦截
对象属性级授权	OWASP API3:2023 - 确保用户只能读取/写入其有权访问的对象属性
PII 泄露	API 响应中意外暴露个人可识别信息，包括姓名、电子邮件、地址、SSN 或财务数据
模式验证	强制 API 响应符合定义的架构，在传输前剥离未授权字段

工具与系统

• Burp Suite Professional：拦截 API 响应并使用 Comparer 工具对比预期与实际响应模式
• mitmproxy：可编写脚本的代理，用于通过基于 Python 的内容检查脚本进行自动化响应分析
• OWASP ZAP：被动扫描器，检测标头、错误消息和响应体中的信息泄露
• Postman：使用测试脚本将文档化的响应模式与实际 API 响应进行对比
• jq：命令行 JSON 处理器，用于从 API 响应中提取和分析特定字段

常见场景

场景：移动银行 API 数据暴露评估

背景：某移动银行应用的 API 向移动客户端返回完整的账户对象，而客户端仅显示账户昵称和余额。该 API 同时被 iOS 和 Android 应用以及一个 Web 门户访问。

方法：

1. 在测试设备上配置 mitmproxy，并以测试用户身份进行认证
2. 在完整的用户会话（登录、查看账户、转账、登出）期间捕获所有 API 响应
3. 分析 GET /api/v1/accounts 响应：UI 仅显示 4 个字段，但 API 返回 23 个字段
4. 发现 API 返回了 routing_number、account_holder_ssn_last4、internal_risk_score、kyc_verification_status 和 linked_external_accounts——这些均未在 UI 中显示
5. 分析 GET /api/v1/transactions 响应：API 返回了客户端并不需要的 merchant_id、terminal_id、authorization_code、processor_response 字段
6. 检查 GET /api/v1/users/me：API 返回了 last_login_ip、mfa_backup_codes_remaining、account_officer_name 和 credit_score_band
7. 测试错误响应：使用无效载荷调用 POST /api/v1/transfers，错误消息中返回了 SQL 表名

常见陷阱：

• 仅检查顶层字段，而遗漏了深层嵌套对象中的敏感数据
• 未测试分页响应，后续页面可能包含不同的字段
• 忽略了可能泄露服务器版本、后端技术或内部路由信息的响应头
• 遗漏了错误响应中的数据暴露，这些响应通常包含堆栈跟踪、SQL 查询或内部路径
• 误以为 HTTPS 加密可以防止数据暴露（它保护的是传输过程，而非防止已认证客户端获取数据）

输出格式

## Finding: Excessive Data Exposure in Account and Transaction APIs

**ID**: API-DATA-001
**Severity**: High (CVSS 7.1)
**OWASP API**: API3:2023 - Broken Object Property Level Authorization
**Affected Endpoints**:
  - GET /api/v1/accounts
  - GET /api/v1/transactions
  - GET /api/v1/users/me

**Description**:
The API returns full database objects to the client, including sensitive fields
that are not displayed in the mobile application UI. The mobile app filters
these fields client-side, but they are fully accessible by intercepting the
API response. This exposes SSN fragments, internal risk scores, and KYC
verification data for any authenticated user.

**Excess Fields Discovered**:
- /accounts: routing_number, account_holder_ssn_last4, internal_risk_score,
  kyc_verification_status, linked_external_accounts (18 excess fields total)
- /transactions: merchant_id, terminal_id, authorization_code,
  processor_response (12 excess fields total)
- /users/me: last_login_ip, mfa_backup_codes_remaining, credit_score_band

**Impact**:
An authenticated user can extract sensitive financial data, internal risk
assessments, and PII for their own account that the application is not
intended to reveal. Combined with BOLA vulnerabilities, this data could
be extracted for all users.

**Remediation**:
1. Implement server-side response filtering using DTOs/view models that only include fields needed by the client
2. Use GraphQL field-level authorization or REST response schemas per endpoint per role
3. Remove sensitive fields from API responses at the serialization layer
4. Implement response schema validation in the API gateway to strip undocumented fields
5. Add automated tests that verify response schemas match documentation