mcpField
P
prompt-analysis

prompt-analysis

github.com/git-ai-project数据与数据库2.9k79 次安装1 周前更新
代码仓库

关于 prompt-analysis

用于分析 Git AI 记录的 AI 编码提示词,数据存放在本地 SQLite 数据库中。技能支持 SQL 汇总查询,也可逐条检查提示词内容、采纳情况、清晰度和重复任务模式。

主要功能

  1. 01初始化并查询 prompts.db
  2. 02按作者和时间范围筛选
  3. 03使用 SQL 汇总指标
  4. 04逐条分析提示词消息
  5. 05对比采纳率与提示词属性

适用场景

  1. 01比较不同模型的采纳率
  2. 02按编码任务分类提示词
  3. 03调查采纳率较低的提示词
  4. 04发现可复用的提示词模式

安装

复制下面的命令,在终端中运行。

$npx skills add https://github.com/git-ai-project/git-ai --skill prompt-analysis

安装后在 Claude Code 中输入/prompt-analysis确认已加载。

详细介绍

提示词分析技能

使用本地 prompts.db SQLite 数据库分析 AI 提示词模式。

什么是 Git AI?

Git AI 是一款用于在 git 中追踪 AI 生成的代码和提示词的工具。它会存储:

  • 每次 AI 对话(提示词和回复)
  • 哪些代码行来自 AI,哪些经过人工编辑
  • 接受率(保留了多少 AI 代码,以及修改了多少)
  • 相关提交和作者

此技能会查询这些数据,帮助用户了解自己的 AI 编码模式。

初始化

首先,根据用户的问题确定范围:

用户提及使用的标志
“我的提示词”或未指定(默认值——当前用户、当前仓库)
“团队”“所有人”“所有作者”--all-authors
某个具体人员的姓名--author "<name>"
某个具体时间范围--since <days>(默认值:30)

发现仅限于笔记——git-ai prompts 始终针对当前仓库运行(工作目录必须位于 git 仓库内)。若要分析多个仓库,请分别在每个仓库中运行命令。

运行初始化:

git-ai prompts [flags]

这会在当前目录中创建或更新 prompts.db。

架构参考

prompts 表包含:

  • seq_id - 用于迭代的自增 ID
  • id - 唯一提示词标识符
  • tool - 使用的工具(例如,“claude-code”“cursor”)
  • model - 模型名称(例如,“claude-sonnet-4-20250514”)
  • human_author - 创建提示词的 Git 用户
  • commit_sha - 相关提交(如果有)
  • total_additions、total_deletions - 更改的代码行数
  • accepted_lines、overridden_lines - 保留的行数与人工修改的行数
  • accepted_rate - 比率:accepted / (accepted + overridden)
  • messages - 对话的 JSON 数组
  • start_time、last_time - Unix 时间戳

分析方法

针对汇总类问题(指标、比较)

使用直接 SQL 查询:

git-ai prompts exec "SELECT model, AVG(accepted_rate), COUNT(*) FROM prompts GROUP BY model"
针对单条提示词分析(分类、内容分析)

当问题需要检查每条提示词的内容(messages JSON)时,请使用子 Agent:

  1. 在架构中添加分析列:
git-ai prompts exec "ALTER TABLE prompts ADD COLUMN work_type TEXT"
git-ai prompts exec "ALTER TABLE prompts ADD COLUMN analysis_notes TEXT"
  1. 重置迭代指针:
git-ai prompts reset
  1. 使用子 Agent 进行迭代——使用 subagent_type: "general-purpose",通过 Task 工具启动并行子 Agent。每个子 Agent:

    • 运行 git-ai prompts next,获取一条提示词的 JSON
    • 分析 messages 内容
    • 更新数据库:git-ai prompts exec "UPDATE prompts SET work_type='...' WHERE id='...'"
  2. 最终综合——查询已补充的数据:

git-ai prompts exec "SELECT work_type, COUNT(*), AVG(accepted_rate) FROM prompts GROUP BY work_type"

子 Agent 迭代模式

逐条处理提示词时,并行启动多个子 Agent。每个子 Agent 的提示词应包含:

Run `git-ai prompts next` to get the next prompt.

Analyze the messages JSON to determine: [specific analysis task]

Then update the database:
git-ai prompts exec "UPDATE prompts SET [column]='[value]' WHERE id='[prompt_id]'"

Return your analysis result.

每次启动 3-5 个子 Agent,检查结果,然后继续启动,直到 git-ai prompts next 返回“No more prompts”。

重要:git-ai prompts next 命令会以 JSON 形式返回分析所需的全部数据,包括:

  • 包含完整对话(人工提示词和 AI 回复)的完整 messages 数组
  • model、tool、accepted_rate、accepted_lines 等元数据

子 Agent 不应运行 git show 或 git log 等其他命令——分析所需的一切都包含在 git-ai prompts next 的 JSON 输出中。请明确告知子 Agent:

IMPORTANT: All data you need is in the JSON output from `git-ai prompts next`.
Do NOT run git commands. Analyze the `messages` field in the JSON directly.

迭代示例

示例 1:按工作类型对提示词分类

用户提问:“按工作类型(错误修复、功能、重构、文档)对我的提示词分类”

设置:

git-ai prompts exec "ALTER TABLE prompts ADD COLUMN work_type TEXT"
git-ai prompts reset

子 Agent 提示词:

Run `git-ai prompts next` to get the next prompt as JSON.

IMPORTANT: All data you need is in this JSON output. Do NOT run git commands.
Analyze the `messages` field directly.

Read the messages JSON and categorize this prompt into ONE of:
- "bug_fix" - fixing broken behavior, errors, or regressions
- "feature" - adding new functionality
- "refactor" - restructuring code without changing behavior
- "docs" - documentation, comments, READMEs
- "test" - adding or modifying tests
- "config" - configuration, build, CI/CD changes
- "other" - doesn't fit above categories

Update the database:
git-ai prompts exec "UPDATE prompts SET work_type='<category>' WHERE id='<prompt_id>'"

Return: the prompt id, your categorization, and a one-sentence reason.

综合查询:

SELECT work_type, COUNT(*) as count,
       ROUND(AVG(accepted_rate), 3) as avg_acceptance,
       SUM(accepted_lines) as total_lines
FROM prompts
WHERE work_type IS NOT NULL
GROUP BY work_type
ORDER BY count DESC
示例 2:分析提示词接受率较低的原因

用户提问:“为什么我的一些提示词接受率较低?”

设置:

git-ai prompts exec "ALTER TABLE prompts ADD COLUMN low_acceptance_reason TEXT"
git-ai prompts exec "UPDATE pointers SET current_seq_id = (SELECT MIN(seq_id) - 1 FROM prompts WHERE accepted_rate < 0.5 AND accepted_rate IS NOT NULL)"

子 Agent 提示词:

Run `git-ai prompts next` to get the next prompt as JSON.

IMPORTANT: All data you need is in this JSON output. Do NOT run git commands.
Analyze the `messages` field directly.

This prompt had a low acceptance rate (human modified most of the AI's code).
Analyze the messages JSON and identify the likely reason:
- "vague_request" - the prompt was unclear or underspecified
- "wrong_approach" - AI took a fundamentally wrong approach
- "style_mismatch" - code worked but didn't match project conventions
- "partial_solution" - AI only solved part of the problem
- "overengineered" - AI added unnecessary complexity
- "context_missing" - AI lacked necessary context about the codebase
- "other" - explain briefly

Update the database:
git-ai prompts exec "UPDATE prompts SET low_acceptance_reason='<reason>' WHERE id='<prompt_id>'"

Return: prompt id, the reason, and specific evidence from the conversation.
示例 3:找出可转化为可复用模式的提示词

用户提问:“我的哪些提示词解决了我可能再次遇到的问题?”

设置:

git-ai prompts exec "ALTER TABLE prompts ADD COLUMN reusable_pattern TEXT"
git-ai prompts exec "ALTER TABLE prompts ADD COLUMN pattern_description TEXT"
git-ai prompts reset

子 Agent 提示词:

Run `git-ai prompts next` to get the next prompt as JSON.

IMPORTANT: All data you need is in this JSON output. Do NOT run git commands.
Analyze the `messages` field directly.

Analyze whether this prompt represents a reusable pattern worth saving:
- Look for: common coding tasks, useful abstractions, clever solutions
- Skip: one-off fixes, highly context-specific changes, trivial edits

If reusable, set reusable_pattern to a short name (e.g., "api_error_handling", "form_validation", "test_mocking")
and pattern_description to a one-sentence description of what it does.

If not reusable, set both to NULL.

git-ai prompts exec "UPDATE prompts SET reusable_pattern='<name>', pattern_description='<desc>' WHERE id='<prompt_id>'"

Return: prompt id and whether you marked it as reusable (with the pattern name if yes).
示例 4:评估提示词的质量和清晰度

用户提问:“我的提示词有多清晰?哪些提示词可以写得更好?”

设置:

git-ai prompts exec "ALTER TABLE prompts ADD COLUMN clarity_score INTEGER"
git-ai prompts exec "ALTER TABLE prompts ADD COLUMN clarity_feedback TEXT"
git-ai prompts reset

子 Agent 提示词:

Run `git-ai prompts next` to get the next prompt as JSON.

IMPORTANT: All data you need is in this JSON output. Do NOT run git commands.
Analyze the `messages` field directly.

Score the HUMAN's prompt clarity from 1-5:
5 = Crystal clear: specific goal, context provided, constraints stated
4 = Good: clear intent, minor ambiguities
3 = Adequate: understandable but missing helpful context
2 = Vague: required AI to make significant assumptions
1 = Unclear: AI had to guess what was wanted

Also provide brief feedback on how the prompt could be improved.

git-ai prompts exec "UPDATE prompts SET clarity_score=<1-5>, clarity_feedback='<feedback>' WHERE id='<prompt_id>'"

Return: prompt id, score, and your feedback.

综合查询:

SELECT clarity_score, COUNT(*) as count,
       ROUND(AVG(accepted_rate), 3) as avg_acceptance
FROM prompts
WHERE clarity_score IS NOT NULL
GROUP BY clarity_score
ORDER BY clarity_score DESC
示例 5:分析提示技巧与接受率的相关性

用户提问:“分析我的提示技巧与接受率之间的相关性”

设置:

git-ai prompts exec "ALTER TABLE prompts ADD COLUMN technique TEXT"
git-ai prompts exec "ALTER TABLE prompts ADD COLUMN technique_notes TEXT"
git-ai prompts reset

子 Agent 提示词:

Run `git-ai prompts next` to get the next prompt as JSON.

IMPORTANT: All data you need is in this JSON output. Do NOT run git commands.
Analyze the `messages` field directly.

Analyze the HUMAN's prompting technique in the messages. Identify which techniques were used:
- "example_driven" - provided examples of desired output or behavior
- "step_by_step" - broke down the request into steps or phases
- "context_heavy" - provided extensive background/context about the codebase
- "minimal" - terse, brief request with little context
- "iterative" - built up solution through back-and-forth refinement
- "constraint_focused" - emphasized what NOT to do or specific requirements
- "reference_based" - pointed to existing code/files to follow as patterns
- "multiple" - combined several techniques (list them in notes)

Also note any specific effective or ineffective patterns in technique_notes.

git-ai prompts exec "UPDATE prompts SET technique='<technique>', technique_notes='<notes>' WHERE id='<prompt_id>'"

Return: prompt id, technique identified, the acceptance_rate, and your observations.

综合查询:

SELECT technique,
       COUNT(*) as count,
       ROUND(AVG(accepted_rate), 3) as avg_acceptance,
       ROUND(MIN(accepted_rate), 3) as min_acceptance,
       ROUND(MAX(accepted_rate), 3) as max_acceptance
FROM prompts
WHERE technique IS NOT NULL AND accepted_rate IS NOT NULL
GROUP BY technique
ORDER BY avg_acceptance DESC
示例 6:对失败的提示词进行定性分析

用户提问:“对失败的提示词(接受率为 0)进行定性分析”

“失败的提示词”的定义:

  • accepted_lines IS NULL - 进行了对话,但没有代码被提交
  • accepted_lines = 0 - 生成了代码,但被人类全部拒绝或重写

设置:

git-ai prompts exec "ALTER TABLE prompts ADD COLUMN failure_analysis TEXT"
git-ai prompts exec "ALTER TABLE prompts ADD COLUMN failure_category TEXT"
git-ai prompts exec "ALTER TABLE prompts ADD COLUMN improvement_suggestion TEXT"
-- Set pointer to iterate only over failed prompts
git-ai prompts exec "DELETE FROM pointers WHERE name='default'"
git-ai prompts exec "INSERT INTO pointers (name, current_seq_id) VALUES ('default', (SELECT MIN(seq_id) - 1 FROM prompts WHERE accepted_lines IS NULL OR accepted_lines = 0))"

子 Agent 提示词:

Run `git-ai prompts next` to get the next prompt as JSON.

IMPORTANT: All data you need is in this JSON output. Do NOT run git commands.
Analyze the `messages` field directly.

This prompt resulted in 0% acceptance - the human rejected or completely rewrote all AI-generated code.

Perform a qualitative analysis by reading the full conversation in messages JSON:

1. **What was requested?** Summarize the human's goal.
2. **What did the AI produce?** Summarize what code/solution was generated.
3. **Why did it fail?** Categorize into:
   - "misunderstood_intent" - AI solved the wrong problem
   - "poor_code_quality" - bugs, errors, or broken code
   - "wrong_technology" - used wrong framework/library/approach
   - "incomplete" - only partially addressed the request
   - "style_violation" - worked but violated project conventions
   - "overcomplicated" - solution far more complex than needed
   - "security_issue" - introduced vulnerabilities
   - "abandoned" - user changed direction mid-conversation

4. **How could the prompt be improved?** Specific, actionable suggestion.

git-ai prompts exec "UPDATE prompts SET failure_category='<category>', failure_analysis='<analysis>', improvement_suggestion='<suggestion>' WHERE id='<prompt_id>'"

Return a detailed analysis including: prompt id, what was requested, what went wrong, and how to prompt better next time.

综合分析:

SELECT failure_category,
       COUNT(*) as count,
       GROUP_CONCAT(id, ', ') as prompt_ids
FROM prompts
WHERE failure_category IS NOT NULL
GROUP BY failure_category
ORDER BY count DESC

然后查看各个 failure_analysis 和 improvement_suggestion 的值,总结经验教训。

示例 7:根据最佳实践对提示词评分

用户提问:“根据最佳实践为我的提示词评分”

设置:

git-ai prompts exec "ALTER TABLE prompts ADD COLUMN grade TEXT"
git-ai prompts exec "ALTER TABLE prompts ADD COLUMN grade_breakdown TEXT"
git-ai prompts exec "ALTER TABLE prompts ADD COLUMN grade_feedback TEXT"
git-ai prompts reset

子 Agent 提示词:

Run `git-ai prompts next` to get the next prompt as JSON.

IMPORTANT: All data you need is in this JSON output. Do NOT run git commands.
Analyze the `messages` field directly.

Grade the HUMAN's prompt against these best practices for AI-assisted coding:

**Grading Criteria (each worth 0-2 points):**

1. **Specificity (0-2)**: Does the prompt clearly state what needs to be done?
   - 0: Vague or ambiguous ("fix this", "make it better")
   - 1: General direction but missing details ("add validation")
   - 2: Specific outcome described ("add email validation that checks format and shows inline error")

2. **Context (0-2)**: Does the prompt provide necessary background?
   - 0: No context, assumes AI knows everything
   - 1: Some context but missing key details
   - 2: Relevant files, constraints, or project patterns mentioned

3. **Scope (0-2)**: Is the request appropriately sized?
   - 0: Too broad ("rewrite the app") or impossibly vague
   - 1: Large but manageable, could be broken down
   - 2: Focused, single-responsibility task

4. **Constraints (0-2)**: Are requirements and boundaries specified?
   - 0: No constraints given when they'd be helpful
   - 1: Some constraints but missing important ones
   - 2: Clear about what to do AND what not to do

5. **Testability (0-2)**: Can success be verified?
   - 0: No way to know if the result is correct
   - 1: Implicit success criteria
   - 2: Explicit expected behavior or acceptance criteria

**Calculate total (0-10) and assign grade:**
- A (9-10): Excellent prompt, follows best practices
- B (7-8): Good prompt, minor improvements possible
- C (5-6): Adequate prompt, notable gaps
- D (3-4): Poor prompt, significant issues
- F (0-2): Ineffective prompt, needs major revision

Store breakdown as JSON: {"specificity": X, "context": X, "scope": X, "constraints": X, "testability": X}

git-ai prompts exec "UPDATE prompts SET grade='<A-F>', grade_breakdown='<json>', grade_feedback='<specific feedback>' WHERE id='<prompt_id>'"

Return: prompt id, grade, total score, and one key improvement suggestion.

综合查询:

SELECT grade,
       COUNT(*) as count,
       ROUND(AVG(accepted_rate), 3) as avg_acceptance,
       SUM(accepted_lines) as total_lines_accepted
FROM prompts
WHERE grade IS NOT NULL AND accepted_rate IS NOT NULL
GROUP BY grade
ORDER BY grade

相关性分析:

-- See if grade correlates with acceptance rate
SELECT
    grade,
    ROUND(AVG(accepted_rate), 3) as avg_acceptance,
    COUNT(*) as n
FROM prompts
WHERE grade IS NOT NULL AND accepted_rate IS NOT NULL
GROUP BY grade
ORDER BY
    CASE grade
        WHEN 'A' THEN 1
        WHEN 'B' THEN 2
        WHEN 'C' THEN 3
        WHEN 'D' THEN 4
        WHEN 'F' THEN 5
    END

查询示例

按模型统计接受率:

SELECT model,
       ROUND(AVG(accepted_rate), 3) as avg_rate,
       SUM(accepted_lines) as total_accepted,
       COUNT(*) as prompt_count
FROM prompts
GROUP BY model
ORDER BY avg_rate DESC

最具成效的提示词(接受率高且代码行数高):

SELECT id, tool, model, accepted_lines, accepted_rate,
       (last_time - start_time) as duration_secs
FROM prompts
WHERE accepted_rate > 0.8 AND accepted_lines > 50
ORDER BY accepted_lines DESC
LIMIT 10

按一天中的时段统计提示词:

SELECT
  CASE
    WHEN (start_time % 86400) / 3600 < 12 THEN 'morning'
    WHEN (start_time % 86400) / 3600 < 17 THEN 'afternoon'
    ELSE 'evening'
  END as time_of_day,
  COUNT(*) as count,
  AVG(accepted_rate) as avg_rate
FROM prompts
GROUP BY time_of_day

清理

prompts.db 文件是临时的。随时删除即可重新开始:

rm prompts.db

故障排除

如果数据库中没有提示词: 此代码库中的 Git AI 可能未正确设置。请指导用户:

重要:Git AI 只会追踪之后的数据。 Git AI 无法追溯归类或追踪安装之前的提示词。只有在设置 Git AI 后进行的提交才会包含提示词数据。如果用户刚安装 Git AI,在积累更多 AI 辅助提交之前,可用数据可能有限。

注意事项和数据说明

并非所有提示词都有统计数据或提交记录:

  • 如果进行了提示词对话,但没有提交代码,commit_sha 为 NULL
  • 对于未提交的工作,accepted_lines、overridden_lines、accepted_rate 可能为 NULL 或 0
  • 出于同样的原因,total_additions、total_deletions 可能为 NULL

原因:

  • 用户进行了对话,但没有接受生成的代码
  • 用户接受了代码,但从未提交
  • 用户仍在进行工作(代码尚未提交)

“失败的提示词”的定义: 分析失败或未成功的提示词时,请使用以下定义:

  • accepted_lines IS NULL - 进行了对话,但没有代码被提交
  • accepted_lines = 0 - 生成了代码,但被人类全部拒绝或重写

查询:WHERE accepted_lines IS NULL OR accepted_lines = 0

在查询中处理:

-- Exclude uncommitted prompts from acceptance rate analysis
SELECT model, AVG(accepted_rate)
FROM prompts
WHERE commit_sha IS NOT NULL AND accepted_rate IS NOT NULL
GROUP BY model

-- Find prompts with conversation but no committed code
SELECT id, tool, start_time
FROM prompts
WHERE commit_sha IS NULL

重要说明

  • messages 列包含 JSON——解析后即可读取对话内容
  • 任何需要读取消息内容的分析都应使用子 Agent
  • SQLite 是临时工作区——可根据分析需要自由添加列
  • 默认范围较保守(当前用户、当前代码库、30 天)
  • 只有用户明确提到团队或所有代码库时,才扩大范围

权限设置

为避免每次运行 git-ai 命令时都收到提示,请将以下内容添加到项目或用户设置中:

项目: .claude/settings.json 用户: ~/.claude/settings.json

{
  "permissions": {
    "allow": [
      "Bash(git-ai:*)"
    ]
  }
}

这对于子 Agent 迭代尤其重要,因为子 Agent 不会继承技能级权限。

评论

支持 Markdown