playwright-screenshot-inspector

Playwright 截图检查器

由大模型驱动的视觉测试专家，使用 Playwright 进行自动截图、分析和 UI 验证，并通过多模态 AI 检查界面。

激活条件

适用于：

• “截图测试”“视觉测试”“截图检查”
• “Playwright 无头模式”“Playwright 截图”
• “UI 验证”“视觉回归”
• “主题合规性测试”“深色模式测试”“浅色模式测试”
• “自动截图”“捕获并分析”
• “比较截图”“视觉差异”

不适用于：

• 简单的一次性截图（使用浏览器 DevTools）
• 不使用 AI 的像素级精确比较（使用原生 Playwright toHaveScreenshot）
• 非 Web UI 测试（使用平台专用工具）
• 性能测试（使用 Lighthouse/WebPageTest）

---

核心理念

传统视觉测试比较像素。大模型驱动的视觉测试能够理解语义。

LLM 检查不再只是回答“这 50 个像素发生了变化”，而是回答：

• “内容是否已实际渲染？”
• “主题是否正确切换？”
• “交互元素是否可见且样式正确？”
• “哪些地方坏了，哪些只是有所不同？”

---

截图检查循环

┌─────────────────────────────────────────────────────────────┐
│                    LLM SCREENSHOT INSPECTION                │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  1. CAPTURE (Playwright)                                    │
│     └─► Wait for React hydration, not just network          │
│                                                             │
│  2. READ (Claude vision)                                    │
│     └─► Pass screenshot to LLM with specific questions      │
│                                                             │
│  3. ANALYZE (Structured response)                           │
│     └─► Extract: content present? theme correct? errors?    │
│                                                             │
│  4. ACT (Conditional logic)                                 │
│     └─► Pass/fail based on semantic understanding           │
│                                                             │
└─────────────────────────────────────────────────────────────┘

---

重要：等待 React 内容加载

最常见的失败原因：在 React 完成水合前就截取屏幕截图。

反面模式：仅等待网络空闲

# ❌ WRONG - React may not have rendered yet
page.goto(url)
page.wait_for_load_state('networkidle')
page.screenshot(path='broken.png')  # Often blank!

正确模式：等待实际内容

# ✅ CORRECT - Wait for React to mount
page.goto(url, wait_until='domcontentloaded')
page.wait_for_load_state('networkidle')

# Give React time to hydrate
import time
time.sleep(0.5)

# Wait for actual content selector
page.wait_for_selector('.main-content, h1, [data-testid="app"]',
                       state='visible',
                       timeout=10000)

# Verify content exists
body_text = page.locator('body').inner_text()
if len(body_text) < 50:
    time.sleep(2)  # Extra wait for slow hydration

page.screenshot(path='good.png', full_page=True)

内容验证函数

def wait_for_react_content(page, selectors, timeout=10000):
    """Wait for React to hydrate by checking for actual content."""
    page.wait_for_load_state('domcontentloaded')
    page.wait_for_load_state('networkidle')
    time.sleep(0.5)  # React hydration buffer

    for selector in selectors.split(','):
        try:
            locator = page.locator(selector.strip())
            if locator.count() > 0:
                locator.first.wait_for(state='visible', timeout=timeout)
                return True
        except:
            continue

    # Fallback: wait for substantial body content
    try:
        page.wait_for_function(
            'document.body.innerText.length > 100',
            timeout=timeout
        )
        return True
    except:
        return False

---

无头模式：避免反复弹出窗口

始终使用 headless=True，避免浏览器窗口不断弹出：

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    # CRITICAL: headless=True prevents visible browser windows
    browser = p.chromium.launch(headless=True)

    context = browser.new_context(
        viewport={'width': 1280, 'height': 800},
        color_scheme='dark'  # Initial theme
    )
    page = context.new_page()

    # ... your test logic ...

    browser.close()  # Always clean up

主题测试模式

# Dark mode screenshot
page.emulate_media(color_scheme='dark')  # Note: on PAGE, not context
page.goto(url)
wait_for_react_content(page, '.app-container, main, h1')
page.screenshot(path='dark.png', full_page=True)

# Light mode screenshot
page.emulate_media(color_scheme='light')
page.reload()
wait_for_react_content(page, '.app-container, main, h1')
page.screenshot(path='light.png', full_page=True)

---

LLM 截图分析模式

模式 1：内容验证

Prompt: "Analyze this screenshot. Answer:
1. Is the main content rendered (not blank/loading)?
2. What major UI elements are visible?
3. Are there any error states or broken layouts?
4. Rate content completeness: FULL / PARTIAL / EMPTY"

模式 2：主题合规性

Prompt: "This is a {dark/light} mode screenshot. Verify:
1. Background color matches expected theme (dark bg for dark mode)
2. Text has sufficient contrast against background
3. Interactive elements are visible and styled correctly
4. No theme leakage (dark elements on light bg or vice versa)"

模式 3：比较分析

Prompt: "Compare these two screenshots (before/after). Identify:
1. What changed between them?
2. Are changes intentional (theme switch) or bugs?
3. Is any content missing in the 'after' version?
4. Rate similarity: IDENTICAL / MINOR_DIFF / MAJOR_DIFF / BROKEN"

模式 4：无障碍检查

Prompt: "Evaluate this screenshot for visual accessibility:
1. Is text readable (sufficient size and contrast)?
2. Are interactive elements clearly identifiable?
3. Is there visual hierarchy (headings, sections)?
4. Any elements that would fail WCAG contrast requirements?"

---

完整测试脚本模板

#!/usr/bin/env python3
"""
LLM-Powered Screenshot Test Suite
Captures screenshots and uses Claude vision for semantic analysis.
"""

from playwright.sync_api import sync_playwright
import os
import time

PAGES_TO_TEST = [
    # (path, name, content_selectors)
    ('/', 'Home', '.hero, main, h1'),
    ('/about', 'About', '.about-content, main, h1'),
    ('/dashboard', 'Dashboard', '.dashboard, .stats, h1'),
]

BASE_URL = 'http://localhost:5173'
SCREENSHOT_DIR = '/tmp/visual-tests'


def wait_for_content(page, selectors, timeout=10000):
    """Wait for React/Vue/Svelte to hydrate."""
    page.wait_for_load_state('domcontentloaded')
    page.wait_for_load_state('networkidle')
    time.sleep(0.5)

    for selector in selectors.split(','):
        try:
            loc = page.locator(selector.strip())
            if loc.count() > 0:
                loc.first.wait_for(state='visible', timeout=timeout)
                return True
        except:
            continue

    try:
        page.wait_for_function('document.body.innerText.length > 100', timeout=timeout)
        return True
    except:
        return False


def capture_themed_screenshots(page, url, name, selectors):
    """Capture both dark and light mode screenshots."""
    safe_name = name.lower().replace(' ', '-')
    results = {'name': name, 'url': url}

    for theme in ['dark', 'light']:
        page.emulate_media(color_scheme=theme)

        if theme == 'dark':
            page.goto(url, wait_until='domcontentloaded')
        else:
            page.reload(wait_until='domcontentloaded')

        content_loaded = wait_for_content(page, selectors)

        if not content_loaded:
            print(f"  ⚠️  {theme} mode: Content slow to load, waiting...")
            time.sleep(2)

        screenshot_path = f'{SCREENSHOT_DIR}/{safe_name}-{theme}.png'
        page.screenshot(path=screenshot_path, full_page=True)

        # Check content length
        body_text = page.locator('body').inner_text().strip()
        results[f'{theme}_screenshot'] = screenshot_path
        results[f'{theme}_content_length'] = len(body_text)
        results[f'{theme}_has_content'] = len(body_text) > 50

        print(f"  {theme}: {'✅' if results[f'{theme}_has_content'] else '❌'} ({len(body_text)} chars)")

    return results


def run_tests():
    """Run visual tests on all pages."""
    os.makedirs(SCREENSHOT_DIR, exist_ok=True)

    with sync_playwright() as p:
        browser = p.chromium.launch(headless=True)
        context = browser.new_context(
            viewport={'width': 1280, 'height': 800},
            color_scheme='dark'
        )
        page = context.new_page()

        # Capture console errors
        errors = []
        page.on('console', lambda m: errors.append(m.text) if m.type == 'error' else None)

        results = []

        for path, name, selectors in PAGES_TO_TEST:
            print(f"Testing {name}...")
            url = f'{BASE_URL}{path}'
            result = capture_themed_screenshots(page, url, name, selectors)
            result['errors'] = list(errors)
            errors.clear()
            results.append(result)

        browser.close()

        # Summary
        print("\n" + "=" * 50)
        print("VISUAL TEST SUMMARY")
        print("=" * 50)

        passed = sum(1 for r in results
                     if r.get('dark_has_content') and r.get('light_has_content'))
        print(f"\nPassed: {passed}/{len(results)}")
        print(f"Screenshots: {SCREENSHOT_DIR}")

        return results


if __name__ == '__main__':
    run_tests()

---

MCP 与原生 Playwright 决策树

What are you doing?
│
├─ Interactive debugging / exploring
│  └─► Playwright MCP (see live browser)
│
├─ Automated test suite
│  └─► Native Python Playwright (headless)
│
├─ CI/CD pipeline
│  └─► Native Python Playwright (headless)
│
├─ Screenshot capture for LLM analysis
│  └─► Native Python Playwright (headless)
│
└─ One-off inspection
   └─► Either works, MCP is convenient

---

常见故障及修复方法

故障：截图为空白

原因：在 React 完成水合前就截取了屏幕截图
修复方法：等待内容选择器，并增加水合缓冲时间

故障：显示“正在重新连接……”徽标

原因：HMR/WebSocket 尚未连接（在测试中属于外观问题）
修复方法：这通常没有问题，应关注实际内容

故障：主题未生效

原因：在 context 而非 page 上调用了 emulate_media
修复方法：使用 page.emulate_media(color_scheme='dark')

故障：浏览器窗口不断弹出

原因：headless=False，或使用了 MCP 而非原生方式
修复方法：使用 p.chromium.launch(headless=True)

故障：等待内容超时

原因：选择器错误，或页面本身已损坏
修复方法：确认选择器存在，并检查控制台错误

---

与 Claude Code 集成

Claude 读取按照此模式捕获的屏幕截图时：

1. 提出具体的分析要求：不要只展示截图，要提出有针对性的问题
2. 提供上下文：“这应该是深色模式”或“这是登录页面”
3. 系统地进行比较：前后对比、深色与浅色对比、桌面与移动端对比
4. 相信语义分析：LLM 能区分“空白页面”和“内容已加载”

---

参考资料

研究论文

• 使用视觉 LLM 进行 UI 测试 - 华盛顿大学
• 视觉驱动的自动化移动 GUI 测试 - 多模态 LLM 方法
• ScreenLLM：有状态的屏幕架构 - UI 理解框架

工具与集成

• 使用 Claude + Playwright 构建 AI QA 工程师
• Playwright 中由 AI 驱动的视觉测试
• Playwright 视觉回归测试指南

官方文档

• Playwright 视觉比较

---

版本历史

• 2026-01-23：首次创建技能
  • 调研多模态大模型分析截图的最佳实践
  • 记录 React hydration 等待模式
  • 添加无头模式要求
  • 创建完整的测试脚本模板

---

核心见解：无用的截图测试与有用的截图测试之间的区别在于等待内容，而不仅仅是等待网络。大模型可以分析语义，但前提是确实有内容可供分析。