Llmcache

llmcache

Stop paying Claude to understand the same code it already understood yesterday.

llmcache is an MCP server that caches AI-generated understanding of your source code. Claude reads each file once and every session after that starts from cached understanding, saving you tokens and money.

Why It Matters

Claude Code uses a stateless API. On every turn, the entire conversation history, including every file Claude has read, is re-sent as input tokens. A 1,000-token file read on turn 1 costs you 1,000 input tokens on every subsequent turn for the rest of the session.

llmcache replaces that 1,000-token file with a 50-token summary. You save 950 tokens on every turn, not just once.

On top of that, Claude generates output tokens to process each file. With a cached summary, Claude skips that work entirely. Output tokens cost 5x more than input tokens.

Without llmcache:
  Turn 1: prompt + file.go (1,000 tokens)        > you pay for 1,000
  Turn 5: prompt + file.go still in history       > you pay for 1,000 again
  Turn 10: still carrying it                      > and again

With llmcache:
  Turn 1: prompt + summary (50 tokens)            > you pay for 50
  Turn 5: summary still in history                > you pay for 50
  Turn 10: still just 50 tokens                   > 95% smaller

Prerequisites

• Claude Code (only supported client)
• Git (used for cache invalidation)
• One of:
  • Anthropic API key (ANTHROPIC_API_KEY)
  • GCP project with Vertex AI API enabled + gcloud auth application-default login

Install

Download the binary for your platform:

# macOS (Apple Silicon)
curl -L -o llmcache https://github.com/sp98/llmcache-site/releases/latest/download/llmcache-darwin-arm64

# macOS (Intel)
curl -L -o llmcache https://github.com/sp98/llmcache-site/releases/latest/download/llmcache-darwin-amd64

# Linux (x86_64)
curl -L -o llmcache https://github.com/sp98/llmcache-site/releases/latest/download/llmcache-linux-amd64

chmod +x llmcache
sudo mv llmcache /usr/local/bin/

How It Works

1. Claude Code calls understand_file (via MCP) instead of Read
2. llmcache checks if a cached summary exists and is still fresh (git hash match)
3. Cache hit: returns the compact summary (typically 90%+ smaller than the full file)
4. Cache miss: calls Claude Haiku to generate a summary, caches it, returns it

Summaries auto-invalidate when the file's git hash changes.

Setup

Local cache stored in SQLite. No server, no license key needed.

1. Add the MCP server

claude mcp add --transport stdio llmcache -- llmcache mcp

For Vertex AI users, pass the region:

claude mcp add \
  --env CLOUD_ML_REGION=us-east5 \
  --transport stdio llmcache -- llmcache mcp

2. Add the Read hook

This hook tells Claude to use understand_file instead of reading source files directly. Add to ~/.claude/settings.json:

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Read",
        "hooks": [
          {
            "type": "command",
            "command": "F=$(cat | grep -o '\"file_path\":\"[^\"]*\"' | head -1 | cut -d'\"' -f4); B=${F##*/}; E=${F##*.}; case \"$B\" in Makefile|makefile|GNUmakefile) echo '{\"permissionDecision\":\"deny\",\"permissionDecisionReason\":\"Use understand_file from llmcache instead.\"}';; *) case \"$E\" in go|py|js|ts|tsx|jsx|rs|java|c|cpp|cc|h|hpp|rb|sh|proto|sql|md) echo '{\"permissionDecision\":\"deny\",\"permissionDecisionReason\":\"Use understand_file or understand_package from llmcache instead.\"}';; esac;; esac"
          }
        ]
      }
    ]
  }
}

3. Add the CLAUDE.md instruction

Append to ~/.claude/CLAUDE.md:

MANDATORY: ALWAYS use understand_file or understand_package from the
llmcache MCP server instead of Read to understand source files.
Only use Read when you need exact line numbers for editing.

4. Verify

Restart Claude Code, use it on a project, then check savings:

llmcache stats
llmcache stats --detail    # per-file breakdown
llmcache stats --monthly   # month-by-month

Configuration

Optional settings via env vars or ~/.llmcache/config.yaml:

Env Var	Default	Description
LLMCACHE_DB	~/.llmcache/data.db	SQLite database path
LLMCACHE_LOG_DIR	~/.llmcache/	Log directory
LLMCACHE_SUMMARY_MODEL	claude-haiku-4-5@20251001	Model for generating summaries

Team Mode (Coming Soon)

In Solo mode, each developer builds their own cache from scratch. When developer A understands a file, developer B still pays full tokens to understand the same file. With Team mode, summaries are shared: one developer pays the generation cost, everyone else gets instant cache hits. The more developers on the team, the faster the cache warms up and the greater the savings.

CLI Commands

Command	Description
llmcache mcp	Start the MCP server (spawned automatically by Claude Code).
llmcache stats	Show cache hits, input/output tokens saved. Use --detail for per-file breakdown, --monthly for trends.
llmcache index <repo>	Pre-index a repository so the first session starts with a warm cache. Use -w 10 for parallel workers, --include "*.go" to filter.
llmcache gc	Remove stale summaries. Use --max-age 720h to set retention, --dry-run to preview.

MCP Tools (used by Claude Code automatically)

Tool	Description
understand_file	Returns a cached summary of a source file. If the file has changed (git hash mismatch), generates a fresh summary. If uncommitted, tells Claude to read directly.
understand_package	Summarizes all source files in a directory, giving Claude a package-level overview in one call.
get_file_detail	Extracts a specific function or type from a file. Uses Go AST for Go files, pattern search for others.
list_summaries	Lists all cached summaries for a repository with hit counts and freshness.

Bugs & Feedback

Found a bug or have a feature request? Open an issue on GitHub.