mcpField
A
arxiv-analyze

arxiv-analyze

github.com/dsebastienProductivity780 installsUpdated 28w ago
Repository

About arxiv-analyze

arxiv-analyze fetches and analyzes arXiv papers for people reading research literature. It tries multiple formats in sequence and caches fetched content for later, offline reads.

Key features

  1. 01Fetch papers through a format fallback chain
  2. 02Support Markdown, HTML, PDF, and TeX
  3. 03Cache content for offline rereading
  4. 04Produce structured paper analyses

Use cases

  1. 01Fetch and extract an arXiv paper
  2. 02Analyze a paper’s methods, results, and limitations
  3. 03Fall back across paper format retrieval options

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/dsebastien/ai-skill-arxiv --skill arxiv-analyze

After installing, type/arxiv-analyzein Claude Code to confirm it loaded.

Bundled filesscripts/arxiv_fetch.py

Details

From the project README

arXiv Analyze

Fetch an arXiv paper in the most token-efficient format available, then analyze it.

Fallback chain

The fetcher script tries formats in order of quality and token efficiency:

  1. arxiv2md (markdown) — cleanest, LaTeX math preserved, refs/TOC/citations stripped by default; rate-limited client-side to 28 req/min
  2. arxiv.org/html — official HTML, can be auto-converted to markdown
  3. ar5iv — arXiv Labs HTML renderer, broader coverage for older papers
  4. arxiv.org/pdf — last resort; URL returned, load with your PDF reader of choice

Opt-in tier 5: TeX source (--tier tex) — downloads arxiv.org/src/<id> (the original submitted tarball), extracts it to the cache, auto-detects the main entrypoint (main.tex / \documentclass / largest .tex), and recursively flattens all \input/\include/\import/\subimport directives. This is the canonical ground truth: available for every paper (even those without HTML renderings), equations are pristine, no third-party dependency. Use it when the paper is math-heavy, when rendered tiers fail, or when you need bibliography entries. Token cost is higher than markdown — prefer auto-tier for most reads.

Disk cache

Fetched content is cached at $XDG_CACHE_HOME/ai-skill-arxiv/<arxiv_id>/ (defaults to ~/.cache/ai-skill-arxiv/<arxiv_id>/). Subsequent fetches of the same paper + tier are instant and offline-safe. Re-reads cost zero network and zero rate-limit budget.

Structure:

~/.cache/ai-skill-arxiv/2501.11120/
├── markdown.md          # arxiv2md output
├── arxiv.html           # arxiv.org/html
├── ar5iv.html           # ar5iv
├── src.tar.gz           # TeX source tarball (only if --tier tex was used)
└── tex/                 # Extracted TeX source tree
    ├── main.tex
    └── ...

Control:

python3 scripts/arxiv_fetch.py <id> --no-cache          # skip cache read, still write
python3 scripts/arxiv_fetch.py --cache-clear <id>       # wipe cache for one paper

Rate limiting (deterministic)

arxiv2md allows 30 requests/minute per IP. The script self-throttles at 28 via scripts/arxiv2md_ratelimit.json — an array of timestamps, pruned to the rolling 60-second window on every call, written atomically (temp file + os.replace). When the limit is reached, the script silently falls through to tier 2. Never edit that JSON by hand; to reset, delete the file.

Check state any time:

python3 scripts/arxiv_fetch.py --ratelimit-status

Usage

# Auto-tier fallback (preferred)
python3 scripts/arxiv_fetch.py 2501.11120

# Full URLs are accepted too
python3 scripts/arxiv_fetch.py "https://arxiv.org/abs/2501.11120v1"

# Force a specific tier
python3 scripts/arxiv_fetch.py 2501.11120 --tier md
python3 scripts/arxiv_fetch.py 2501.11120 --tier html
python3 scripts/arxiv_fetch.py 2501.11120 --tier ar5iv
python3 scripts/arxiv_fetch.py 2501.11120 --tier pdf
python3 scripts/arxiv_fetch.py 2501.11120 --tier tex    # TeX source (opt-in)

# Metadata only (arXiv API Atom feed, ~200 tokens)
python3 scripts/arxiv_fetch.py 2501.11120 --metadata-only

# Cache control
python3 scripts/arxiv_fetch.py 2501.11120 --no-cache    # force fresh fetch
python3 scripts/arxiv_fetch.py --cache-clear 2501.11120 # wipe cache for one id

The selected tier is printed to stderr ([tier] md); content goes to stdout. For the PDF tier, stdout is the URL — feed it to your PDF reader.

Exit codes: 0 success, 1 all-tiers-failed, 2 forced-tier-unavailable, 3 invalid-ID, 4 network-error.

Analysis workflow

After fetching, produce a structured summary with these sections:

  • Citation — authors, title, arXiv ID, year, primary category
  • Problem — what question the paper answers; why it matters
  • Key claims — 3-5 bullets, each a specific claim (not a heading)
  • Method — how they approached it, in 2-4 sentences
  • Results — what they found; include numbers that matter
  • Limitations — what the authors admit + what you noticed
  • Open questions — what's unresolved; what follow-up work would do

Token budget

TierTypical outputWhen to use
metadata-only~200 tokensPreview before committing to a full fetch
md (arxiv2md)5K-15KDefault; most efficient
html (arxiv.org)8K-20Karxiv2md unavailable or rate-limited
ar5iv8K-20KOlder papers without official HTML
pdf (via Read)10K-50K+Equations-heavy papers where HTML rendering is bad
tex (opt-in)15K-60K+Ground truth (no third-party dependency); math-heavy papers; auditing references; always-available fallback

If you only need claims/results, dump the fetched content to a temp file and grep/head rather than passing the whole thing to context. The paper is on disk — don't re-fetch.

Hard rules

  • Never fabricate results. If a claim isn't in the paper, say so.
  • Respect the rate limit. The script handles it automatically; don't loop --tier md manually.
  • PDFs are expensive. Only use --tier pdf when HTML paths have all failed or the user explicitly requests it.

Requirements

  • Python 3.11+ (stdlib only, no pip install needed)

Credits

Comments

Markdown supported