mcpField
L
log-aggregation

log-aggregation

github.com/aj-geddesCloud & DevOps373449 installsUpdated 34w ago
Repository

About log-aggregation

A skill for designing log aggregation systems that collect, structure, and analyze logs from multiple sources. It provides implementation guidance for centralized monitoring, debugging, security, and audit needs.

Key features

  1. 01Collect and parse logs from multiple sources
  2. 02ELK Stack and Logstash setup guides
  3. 03Filebeat and Loki configuration
  4. 04Kibana dashboards and alerts
  5. 05Log retention and access-control practices

Use cases

  1. 01Centralize logs across distributed services
  2. 02Debug application errors using searchable logs
  3. 03Monitor security events and compliance audits

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/aj-geddes/useful-ai-prompts --skill log-aggregation

After installing, type/log-aggregationin Claude Code to confirm it loaded.

Bundled files

References and scripts installed along with SKILL.md.

references/
elk-stack-configuration.md
filebeat-configuration.md
kibana-dashboard-and-alerts.md
log-aggregation-deployment-script.md
logstash-pipeline-configuration.md
loki-configuration-kubernetes.md
scripts/
validate-config.sh
templates/
config-starter.yaml

Details

From the project README

Log Aggregation

Table of Contents

Overview

Build comprehensive log aggregation systems to collect, parse, and analyze logs from multiple sources, enabling centralized monitoring, debugging, and compliance auditing.

When to Use

  • Centralized log collection
  • Distributed system debugging
  • Compliance and audit logging
  • Security event monitoring
  • Application performance analysis
  • Error tracking and alerting
  • Historical log retention
  • Real-time log searching

Quick Start

Minimal working example:

# docker-compose.yml - ELK Stack setup
version: "3.8"

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.5.0
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ports:
      - "9200:9200"
    volumes:
      - elasticsearch_data:/usr/share/elasticsearch/data
    healthcheck:
      test: curl -s http://localhost:9200 >/dev/null || exit 1
      interval: 10s
      timeout: 5s
      retries: 5

  logstash:
    image: docker.elastic.co/logstash/logstash:8.5.0
    volumes:
      - ./logstash.conf:/usr/share/logstash/pipeline/logstash.conf
    ports:
// ... (see reference guides for full implementation)

Reference Guides

Detailed implementations in the references/ directory:

GuideContents
ELK Stack ConfigurationELK Stack Configuration
Logstash Pipeline ConfigurationLogstash Pipeline Configuration
Filebeat ConfigurationFilebeat Configuration
Kibana Dashboard and AlertsKibana Dashboard and Alerts
Loki Configuration (Kubernetes)Loki Configuration (Kubernetes)
Log Aggregation Deployment ScriptLog Aggregation Deployment Script

Best Practices

✅ DO
  • Parse and structure log data
  • Use appropriate log levels
  • Add contextual information
  • Implement log retention policies
  • Set up log-based alerting
  • Index important fields
  • Use consistent timestamp formats
  • Implement access controls
❌ DON'T
  • Store sensitive data in logs
  • Log at DEBUG level in production
  • Send raw unstructured logs
  • Ignore storage costs
  • Skip log parsing
  • Lack monitoring of log systems
  • Store logs forever
  • Log PII without encryption

Comments

Markdown supported