mcpField
S
sec-edgar-pipeline

sec-edgar-pipeline

github.com/bobmatnycData & databases78346 installsUpdated 15w ago
Repository

About sec-edgar-pipeline

A command-line workflow for extracting and reporting data from SEC EDGAR filings, centered on edgar-analyzer. It guides users through project setup, pattern analysis, extraction, validation, and CSV report generation.

Key features

  1. 01Project setup and example analysis
  2. 02EDGAR filing retrieval workflows
  3. 03Recipe-based table extraction
  4. 04Validation of extracted results
  5. 05JSON-to-CSV report generation

Use cases

  1. 01Extracting compensation tables from DEF 14A filings
  2. 02Running repeatable filing extraction projects
  3. 03Validating structured filing data
  4. 04Preparing executive compensation CSV reports

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/bobmatnyc/claude-mpm-skills --skill sec-edgar-pipeline

After installing, type/sec-edgar-pipelinein Claude Code to confirm it loaded.

Bundled filesmetadata.json

Details

From the project README

SEC EDGAR Pipeline

Overview

This pipeline is centered on edgar-analyzer and the EDGAR data sources. The core loop is: configure credentials, create a project with examples, analyze patterns, generate code, run extraction, and export reports.

Setup (Keys + User Agent)

Use the setup wizard to configure required keys:

python -m edgar_analyzer setup
# or
edgar-analyzer setup

Required entries:

  • OPENROUTER_API_KEY
  • (Optional) JINA_API_KEY
  • EDGAR user agent string ("Name [email protected]")

End-to-End CLI Workflow

# 1. Create project
edgar-analyzer project create my_project --template minimal

# 2. Add examples + project.yaml
# projects/my_project/examples/*.json

# 3. Analyze examples
edgar-analyzer analyze-project projects/my_project

# 4. Generate extraction code
edgar-analyzer generate-code projects/my_project

# 5. Run extraction
edgar-analyzer run-extraction projects/my_project --output-format csv

Outputs land in projects/<name>/output/.

EDGAR-Specific Conventions

  • CIK values are 10-digit, zero-padded (e.g., 0000320193).
  • Rate limit: SEC API allows 10 requests/sec. Scripts use ~0.11s delays.
  • User agent is mandatory; include name + email.

Scripted Example (Apple DEF 14A)

edgar/scripts/fetch_apple_def14a.py shows the direct flow:

  1. Fetch latest DEF 14A metadata
  2. Download HTML
  3. Parse Summary Compensation Table (SCT)
  4. Save raw HTML + extracted JSON + ground truth

Recipe-Driven Extraction

edgar/recipes/sct_extraction/config.yaml defines a multi-step pipeline:

  • Fetch DEF 14A filings by company list
  • Extract SCT tables with SCTAdapter
  • Validate with sct_validator
  • Write results to output/sct

Report Generation

edgar/scripts/create_csv_reports.py converts JSON results into:

  • executive_compensation_<timestamp>.csv
  • top_25_executives_<timestamp>.csv
  • company_summary_<timestamp>.csv

Troubleshooting

  • No filings found: confirm CIK formatting and filing type (DEF 14A vs DEF 14A/A).
  • API errors: slow down requests and confirm user-agent is set.
  • Extraction errors: regenerate code or use manual ground truth in POC scripts.

Related Skills

  • universal/data/reporting-pipelines
  • toolchains/python/testing/pytest

Comments

Markdown supported