mcpField
P
prometheus

prometheus

github.com/grafana云与 DevOps2955.3k 次安装1 天前更新
代码仓库

关于 prometheus

prometheus 是介绍 Prometheus 指标工作流的技能，涵盖 PromQL、告警和记录规则。它面向查询或运维 Prometheus、Grafana 指标的用户，并提供常见分析与配置示例。

主要功能

  1. 01PromQL 选择器、速率与聚合
  2. 02常见指标分析模式
  3. 03告警规则和路由示例
  4. 04记录规则配置与验证
  5. 05Grafana Metrics Drilldown 指引

适用场景

  1. 01编写请求速率和错误率查询
  2. 02配置并验证 Prometheus 告警
  3. 03用记录规则预计算仪表板指标
  4. 04无需编写 PromQL 浏览指标

安装

复制下面的命令，在终端中运行。

$npx skills add https://github.com/grafana/skills --skill prometheus

安装后在 Claude Code 中输入/prometheus确认已加载。

详细介绍

使用 Prometheus 和 Grafana 监控指标

文档：https://prometheus.io/docs/ | Grafana Cloud Metrics：https://grafana.com/docs/grafana-cloud/send-data/metrics/

PromQL 快速参考

即时向量选择器
# By metric name
http_requests_total

# Label filter
http_requests_total{job="api-server"}

# Multiple labels (AND)
http_requests_total{job="api-server", method="GET"}

# Regex
http_requests_total{job=~"api.*", status=~"5.."}

# Negative
http_requests_total{status!="200"}
范围向量和速率
# Per-second rate over 5 minutes
rate(http_requests_total[5m])

# Increase over interval
increase(http_requests_total[1h])

# Instant rate (last two samples)
irate(http_requests_total[5m])

# Offset (5 minutes ago)
rate(http_requests_total[5m] offset 5m)
聚合
# Sum by label
sum by (job) (rate(http_requests_total[5m]))

# Average
avg by (instance) (node_cpu_seconds_total)

# Top-K
topk(5, rate(http_requests_total[5m]))

# Histogram quantiles
histogram_quantile(0.99, rate(http_request_duration_seconds_bucket[5m]))

# Count distinct
count(up{job="api"})
常见模式
# Error rate percentage
sum(rate(http_requests_total{status=~"5.."}[5m]))
  / sum(rate(http_requests_total[5m])) * 100

# Saturation (CPU usage %)
100 - (avg by(instance) (irate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

# Memory usage
node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes

# Predict disk full (linear extrapolation)
predict_linear(node_filesystem_free_bytes[6h], 24*3600) < 0

告警规则

Prometheus 告警规则
groups:
  - name: api_alerts
    rules:
      - alert: HighErrorRate
        expr: |
          sum(rate(http_requests_total{status=~"5.."}[5m]))
            / sum(rate(http_requests_total[5m])) > 0.05
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "High 5xx error rate ({{ $value | humanizePercentage }})"
Alertmanager 路由
# alertmanager.yml
route:
  receiver: default
  group_by: [alertname, job]
  group_wait: 30s
  group_interval: 5m
  routes:
    - match:
        severity: critical
      receiver: pagerduty
    - match:
        severity: warning
      receiver: slack

receivers:
  - name: pagerduty
    pagerduty_configs:
      - service_key: "<key>"
  - name: slack
    slack_configs:
      - channel: "#alerts"
        api_url: "<webhook_url>"
  - name: default
    email_configs:
      - to: "[email protected]"
验证告警配置
promtool check rules rules.yml
amtool check-config alertmanager.yml
amtool config routes test --config.file=alertmanager.yml severity=critical

记录规则

预先计算开销较大的 PromQL，以提升仪表板性能：

groups:
  - name: api_rules
    interval: 1m
    rules:
      - record: job:http_requests:rate5m
        expr: sum by (job) (rate(http_requests_total[5m]))
      - record: job:http_request_duration_seconds:p99
        expr: histogram_quantile(0.99, sum by (job, le) (rate(http_request_duration_seconds_bucket[5m])))
部署并验证记录规则
# 1. Validate rule syntax
promtool check rules rules/recording.yml

# 2. Reload Prometheus (after adding to rule_files in prometheus.yml)
curl -X POST http://localhost:9090/-/reload

# 3. Verify rules are active
curl -s http://localhost:9090/api/v1/rules | jq '.data.groups[].rules[] | {name, health}'

指标深入分析（Grafana 12+）

无需编写 PromQL 即可探索 Prometheus — 浏览指标。前往 Explore > Metrics Drilldown，或使用 <grafana-url>/a/grafana-metricsdrilldown-app。 提供指标搜索和标签细分、用于异常检测的智能分段、自动可视化，以及从指标切换到相关日志和追踪的遥测数据透视功能。

资源

评论

支持 Markdown