关于 problem-analysis
面向 ServiceNow 问题管理人员的根因分析技能,涵盖从事件识别、调查和已知错误记录,到解决方案跟踪与关闭的流程,并提供 5 Whys 和 Fishbone 分析方法。
主要功能
- 01从事件中识别重复问题
- 02使用 5 Whys 和 Fishbone 分析根因
- 03创建问题调查任务和已知错误记录
- 04记录临时解决方法并关联事件
- 05跟踪永久修复并验证效果
适用场景
- 01分析具有相同根因的多个事件
- 02开展重大事件后的复盘
- 03调查反复出现的服务降级
- 04根据趋势主动识别潜在问题
MCP 相关
该技能通过 MCP 工具查询事件、创建和更新问题及已知错误记录、记录工作备注,并关闭已解决的问题。
安装
复制下面的命令,在终端中运行。
$
npx skills add https:/ / github.com/ happy-technologies-llc/ happy-platform-skills --skill problem-analysis安装后在 Claude Code 中输入/problem-analysis确认已加载。
详细介绍
问题分析与根因调查
概述
此技能为 ServiceNow 中的问题管理提供全面的框架,重点关注识别根因、记录已知错误以及实施永久解决方案。
问题管理目标:
- 识别并消除事件的根因
- 尽量减少无法预防的事件造成的影响
- 在潜在问题导致事件之前主动识别它们
- 记录已知错误和临时解决方法
适用情形:
- 多个事件具有相同根因
- 重大事件后的复盘
- 服务反复降级
- 主动趋势分析
前提条件
- 角色:
problem_manager、problem_admin或itil - 访问权限: 对问题、事件和已知错误表具有读写权限
- 知识: 根因分析技术(5 Whys、鱼骨图)
- 相关技能:
itsm/incident-lifecycle、itsm/major-incident
操作流程
阶段 1:识别问题
步骤 1.1:从事件中识别问题候选项
查找模式相似的事件:
使用 MCP:
Tool: SN-Query-Table
Parameters:
table_name: incident
query: active=false^stateIN6,7^sys_created_on>=javascript:gs.daysAgoStart(30)
fields: sys_id,number,short_description,category,cmdb_ci,resolution_notes,close_code
limit: 100
使用 REST API:
GET /api/now/table/incident?sysparm_query=active=false^stateIN6,7^sys_created_on>=javascript:gs.daysAgoStart(30)&sysparm_fields=sys_id,number,short_description,category,cmdb_ci,resolution_notes,close_code&sysparm_limit=100
按 CI 查找重复事件:
Tool: SN-Query-Table
Parameters:
table_name: incident
query: cmdb_ci=[ci_sys_id]^sys_created_on>=javascript:gs.daysAgoStart(90)
fields: sys_id,number,short_description,resolution_notes,close_code
order_by: sys_created_on
步骤 1.2:分析事件模式
分组标准:
- 相同的配置项(CI)
- 相同的类别/子类别
- 简短描述相似(关键词匹配)
- 相同的分配组
- 相同的解决类型
模式分析工作备注:
Tool: SN-Add-Work-Notes
Parameters:
sys_id: [analysis_task_sys_id]
work_notes: |
=== INCIDENT PATTERN ANALYSIS ===
Analysis Period: [date range]
PATTERN IDENTIFIED:
- Total Related Incidents: [count]
- Affected CI: [CI name]
- Common Category: [category]
- Keyword Pattern: [keywords]
INCIDENT LIST:
- INC001234 - [date] - [short description]
- INC001235 - [date] - [short description]
- INC001240 - [date] - [short description]
FREQUENCY:
- First occurrence: [date]
- Last occurrence: [date]
- Average frequency: [X per week/month]
RECOMMENDATION:
Create problem record for root cause investigation.
步骤 1.3:创建问题记录
使用 MCP:
Tool: SN-Create-Record
Parameters:
table_name: problem
data:
short_description: "[CI/Service] - [Brief description of recurring issue]"
description: |
PROBLEM STATEMENT:
[Clear description of the problem being investigated]
RELATED INCIDENTS:
- INC001234 - [date] - [description]
- INC001235 - [date] - [description]
- INC001240 - [date] - [description]
BUSINESS IMPACT:
- Number of incidents: [count]
- Total downtime: [hours]
- Users affected: [count]
- Business cost: [estimate]
INITIAL HYPOTHESIS:
[Initial theory about root cause]
priority: 2
impact: 2
urgency: 2
category: [category]
cmdb_ci: [ci_sys_id]
assignment_group: [team_sys_id]
使用 REST API:
POST /api/now/table/problem
Content-Type: application/json
{
"short_description": "Email Server - Intermittent connection failures",
"description": "PROBLEM STATEMENT:\nUsers experiencing intermittent email connectivity...",
"priority": "2",
"impact": "2",
"urgency": "2",
"category": "software",
"cmdb_ci": "ci_sys_id",
"assignment_group": "group_sys_id"
}
阶段 2:根因调查
步骤 2.1:创建调查任务
为不同领域创建调查任务:
Tool: SN-Create-Record
Parameters:
table_name: problem_task
data:
parent: [problem_sys_id]
short_description: "RCA - [Area] Investigation"
description: |
Investigate [specific area] as potential root cause.
INVESTIGATION SCOPE:
- [Item 1 to investigate]
- [Item 2 to investigate]
- [Item 3 to investigate]
EXPECTED DELIVERABLES:
- Findings documented in work notes
- Evidence collected (logs, screenshots)
- Recommendation for next steps
assignment_group: [specialist_team]
priority: 2
步骤 2.2:5 Whys 分析
记录在问题工作备注中:
Tool: SN-Add-Work-Notes
Parameters:
sys_id: [problem_sys_id]
work_notes: |
=== 5-WHYS ROOT CAUSE ANALYSIS ===
Analyst: [name]
Date: [date]
PROBLEM STATEMENT:
Email service experiencing intermittent connection failures
WHY 1: Why are connection failures occurring?
Answer: The email server is running out of available connections.
WHY 2: Why is the server running out of connections?
Answer: Connection pool is exhausted due to connections not being released.
WHY 3: Why are connections not being released?
Answer: A memory leak in the email client integration is holding connections.
WHY 4: Why is there a memory leak?
Answer: The integration code doesn't properly handle error conditions.
WHY 5: Why doesn't the code handle error conditions?
Answer: Code review process didn't catch the missing error handling.
ROOT CAUSE:
Missing error handling in email client integration code, combined with
insufficient code review process for integration components.
CONTRIBUTING FACTORS:
- No connection timeout configured
- Monitoring didn't alert on connection pool
- Documentation gap on error handling standards
步骤 2.3:鱼骨图(石川图)分析
Tool: SN-Add-Work-Notes
Parameters:
sys_id: [problem_sys_id]
work_notes: |
=== FISHBONE ANALYSIS ===
Problem: [Problem statement]
PEOPLE:
- [Factor 1]
- [Factor 2]
PROCESS:
- [Factor 1]
- [Factor 2]
TECHNOLOGY:
- [Factor 1]
- [Factor 2]
ENVIRONMENT:
- [Factor 1]
- [Factor 2]
DATA:
- [Factor 1]
- [Factor 2]
EXTERNAL:
- [Factor 1]
- [Factor 2]
PRIMARY ROOT CAUSES:
1. [Root cause from analysis]
2. [Contributing root cause]
步骤 2.4:使用根因更新问题记录
使用 MCP:
Tool: SN-Update-Record
Parameters:
table_name: problem
sys_id: [problem_sys_id]
data:
state: 103 # Root Cause Analysis
root_cause: |
ROOT CAUSE IDENTIFIED:
Primary Root Cause:
[Detailed description of the root cause]
Contributing Factors:
1. [Factor 1]
2. [Factor 2]
3. [Factor 3]
Evidence:
- [Log entry/data supporting conclusion]
- [Test result]
- [Other evidence]
Analysis Method: [5-Whys/Fishbone/Fault Tree/Other]
Analysis Date: [date]
Analyst: [name]
阶段 3:记录已知错误
步骤 3.1:创建已知错误记录
确认根因后,将其记录为已知错误:
使用 MCP:
Tool: SN-Create-Record
Parameters:
table_name: known_error
data:
short_description: "[CI] - [Error description]"
description: |
KNOWN ERROR DESCRIPTION:
[Clear description of the error condition]
SYMPTOMS:
- [Symptom 1]
- [Symptom 2]
- [Symptom 3]
ROOT CAUSE:
[Root cause description]
AFFECTED SERVICES/CIS:
- [Service/CI 1]
- [Service/CI 2]
workaround: |
WORKAROUND INSTRUCTIONS:
When to use: [Condition when workaround applies]
Steps:
1. [Step 1]
2. [Step 2]
3. [Step 3]
Expected Result: [What user should see]
Limitations:
- [Limitation 1]
- [Limitation 2]
Contact [team] if workaround does not resolve the issue.
problem: [problem_sys_id]
cmdb_ci: [ci_sys_id]
u_permanent_fix_planned: true
u_fix_date: [target date]
使用 REST API:
POST /api/now/table/known_error
Content-Type: application/json
{
"short_description": "Email Server - Connection timeout during peak hours",
"description": "KNOWN ERROR DESCRIPTION:\nEmail connections may timeout...",
"workaround": "WORKAROUND INSTRUCTIONS:\n1. Close and reopen email client...",
"problem": "problem_sys_id",
"cmdb_ci": "ci_sys_id"
}
步骤 3.2:将已知错误关联到事件
更新相关事件:
Tool: SN-Update-Record
Parameters:
table_name: incident
sys_id: [incident_sys_id]
data:
problem_id: [problem_sys_id]
work_notes: "Linked to Known Error [KERR#] - Workaround available. See KB article [KB#]."
批量更新多个事件:
Tool: SN-Query-Table
Parameters:
table_name: incident
query: cmdb_ci=[ci_sys_id]^problem_idISEMPTY^stateIN1,2,3
fields: sys_id,number
limit: 50
然后对每个事件执行:
Tool: SN-Update-Record
Parameters:
table_name: incident
sys_id: [each_incident_sys_id]
data:
problem_id: [problem_sys_id]
阶段 4:临时解决方法管理
步骤 4.1:记录临时解决方法
详细记录临时解决方法:
Tool: SN-Add-Work-Notes
Parameters:
sys_id: [problem_sys_id]
work_notes: |
=== WORKAROUND DOCUMENTED ===
WORKAROUND ID: WA-[number]
Effective Date: [date]
Author: [name]
APPLICABILITY:
- Applies to: [specific conditions]
- Does NOT apply to: [exclusions]
PREREQUISITES:
- [Prerequisite 1]
- [Prerequisite 2]
PROCEDURE:
1. [Detailed step 1]
Note: [Important note if applicable]
2. [Detailed step 2]
Expected Result: [What to expect]
3. [Detailed step 3]
VERIFICATION:
- [How to verify workaround worked]
KNOWN LIMITATIONS:
- [Limitation 1]
- [Limitation 2]
ROLLBACK PROCEDURE:
If workaround causes issues:
1. [Rollback step 1]
2. [Rollback step 2]
SUPPORT CONTACT:
If workaround fails, contact [team/person] at [contact info]
步骤 4.2:向服务台传达临时解决方法
Tool: SN-Add-Problem-Comment
Parameters:
sys_id: [problem_sys_id]
comment: |
=== WORKAROUND AVAILABLE FOR SERVICE DESK ===
Problem: [PRB#]
Known Error: [KERR#]
QUICK REFERENCE FOR AGENTS:
Customer Reports: "[Common customer description]"
Solution: [Brief description of workaround]
Steps for Customer:
1. [Simple step 1]
2. [Simple step 2]
3. [Simple step 3]
Escalate if: [Condition for escalation]
Related KB: [KB article link]
阶段 5:永久修复
步骤 5.1:规划永久修复
为修复创建变更请求:
Tool: SN-Create-Record
Parameters:
table_name: change_request
data:
short_description: "Fix: [Problem description]"
description: |
CHANGE PURPOSE:
Implement permanent fix for problem [PRB#]
ROOT CAUSE ADDRESSED:
[Root cause from problem record]
PROPOSED SOLUTION:
[Technical description of fix]
EXPECTED OUTCOME:
- [Outcome 1]
- [Outcome 2]
TESTING PLAN:
- [Test 1]
- [Test 2]
ROLLBACK PLAN:
- [Rollback step 1]
- [Rollback step 2]
type: normal
priority: 2
assignment_group: [development_team]
u_related_problem: [problem_sys_id]
步骤 5.2:跟踪修复进度
Tool: SN-Update-Record
Parameters:
table_name: problem
sys_id: [problem_sys_id]
data:
state: 104 # Fix in Progress
fix: |
PERMANENT FIX PLAN:
Solution: [Description of permanent fix]
Implementation Method:
- Change Request: [CHG#]
- Target Date: [date]
- Implementation Team: [team]
Technical Details:
[Detailed technical fix description]
Validation Criteria:
- [ ] [Criterion 1]
- [ ] [Criterion 2]
- [ ] [Criterion 3]
阶段 6:关闭问题
步骤 6.1:验证修复效果
修复后监控:
Tool: SN-Query-Table
Parameters:
table_name: incident
query: cmdb_ci=[ci_sys_id]^sys_created_on>=javascript:gs.daysAgoStart(14)^problem_id=[problem_sys_id]
fields: sys_id,number,short_description,sys_created_on
limit: 50
记录验证结果:
Tool: SN-Add-Work-Notes
Parameters:
sys_id: [problem_sys_id]
work_notes: |
=== FIX VERIFICATION ===
Verification Period: [date range]
METRICS:
- Incidents before fix: [count] per [period]
- Incidents after fix: [count] per [period]
- Reduction: [percentage]
MONITORING DATA:
- [Metric 1]: [before] → [after]
- [Metric 2]: [before] → [after]
USER FEEDBACK:
- [Feedback item 1]
- [Feedback item 2]
VERIFICATION STATUS: [Pass/Fail/Partial]
RECOMMENDATION: [Close problem/Continue monitoring/Additional action]
步骤 6.2:关闭问题记录
使用 MCP:
Tool: SN-Close-Problem
Parameters:
sys_id: [problem_sys_id]
close_code: Fix Applied
close_notes: |
PROBLEM CLOSURE SUMMARY:
Root Cause: [Summary]
Resolution: [What was done]
Implementation:
- Change: [CHG#]
- Date: [implementation date]
Effectiveness:
- Incident reduction: [percentage]
- Monitoring period: [dates]
- No recurrence confirmed
Documentation:
- Known Error: [KERR#]
- Knowledge Article: [KB#]
Lessons Learned:
- [Lesson 1]
- [Lesson 2]
使用 REST API:
PATCH /api/now/table/problem/{sys_id}
Content-Type: application/json
{
"state": "107",
"close_code": "Fix Applied",
"close_notes": "PROBLEM CLOSURE SUMMARY:\n\nRoot Cause: Memory leak in email integration...",
"resolved_at": "2024-01-15 14:30:00",
"resolved_by": "admin"
}
问题状态参考
┌────────────┐ ┌────────────────┐ ┌─────────────┐
│ New │────►│ Root Cause │────►│ Fix in │
│ (101) │ │ Analysis (103) │ │ Progress │
└────────────┘ └────────────────┘ │ (104) │
│ └──────┬──────┘
▼ │
┌───────────────┐ │
│ Known Error │ │
│ (102) │◄─────────────┘
└───────┬───────┘
│
▼
┌───────────────────────────────┐
│ Resolved │
│ (106) │
└───────────────┬───────────────┘
│
▼
┌───────────────────────────────┐
│ Closed │
│ (107) │
└───────────────────────────────┘
工具使用摘要
MCP 工具
| 工具 | 用途 | 阶段 |
|---|---|---|
SN-List-Problems | 列出现有问题 | 1 |
SN-Query-Table | 查找事件模式、验证修复 | 1、6 |
SN-Create-Record | 创建问题、已知错误和任务 | 1、2、3 |
SN-Update-Record | 更新问题状态、根因和修复信息 | 2、5 |
SN-Add-Work-Notes | 记录分析和发现 | 全部 |
SN-Add-Problem-Comment | 与客户/服务台沟通 | 4 |
SN-Close-Problem | 关闭已解决的问题 | 6 |
REST API 端点
| 端点 | 方法 | 用途 |
|---|---|---|
/api/now/table/problem | GET | 列出问题 |
/api/now/table/problem | POST | 创建问题 |
/api/now/table/problem/{sys_id} | PATCH | 更新问题 |
/api/now/table/problem_task | POST | 创建调查任务 |
/api/now/table/known_error | POST | 创建已知错误 |
/api/now/table/incident | GET | 查询相关事件 |
最佳实践
- 数据驱动: 使用事件数据识别问题,而非依赖假设
- 结构化分析: 始终使用正式的 RCA 技术(5 Whys、鱼骨图)
- 记录一切: 详细记录有助于未来的调查人员
- 优先采用临时解决方案: 在制定永久修复方案期间,先缓解当前问题
- 验证有效性: 未确认修复方案有效前,不要关闭问题
- 知识管理: 将调查结果整理成知识库文章
- 符合 ITIL: 问题管理旨在减少事件数量和影响
故障排除
“未找到相关事件”
原因: 查询条件过于严格,或配置项(CI)有误
解决方案: 扩大日期范围;验证 CI sys_id;尝试按关键词搜索
“分析后根本原因仍不明确”
原因: 数据不足,或存在多个促成因素
解决方案: 收集更多数据;邀请更多主题专家(SME)参与;考虑环境因素
“临时解决方案无效”
原因: 临时解决方案未覆盖所有场景
解决方案: 完善临时解决方案;记录其局限性;为其他场景单独创建临时解决方案
“问题不断重新打开”
原因: 根本原因未得到彻底解决;或同一问题出现了新的变体
解决方案: 检查是否确实为同一问题;对于变体,可能需要新建问题记录
RCA 模板
5 Whys 模板
Problem: [Statement]
Why 1: [Question]
Because: [Answer]
Why 2: [Question based on Why 1 answer]
Because: [Answer]
Why 3: [Question based on Why 2 answer]
Because: [Answer]
Why 4: [Question based on Why 3 answer]
Because: [Answer]
Why 5: [Question based on Why 4 answer]
Because: [Answer - typically root cause]
Root Cause: [Summary]
鱼骨图模板
Problem: _________________________________
Categories:
PEOPLE PROCESS TECHNOLOGY ENVIRONMENT
| | | |
+-- [cause] +-- [cause] +-- [cause] +-- [cause]
+-- [cause] +-- [cause] +-- [cause] +-- [cause]
Root Causes Identified:
1. [Primary root cause]
2. [Secondary root cause]
相关技能
itsm/incident-lifecycle- 事件管理itsm/incident-triage- 事件分诊itsm/major-incident- 重大事件处理itsm/change-management- 通过变更实施永久修复knowledge/article-generation- 将调查结果整理成知识库文章
参考资料
评论
支持 Markdown