About Backtest Overfitting Audit
An MCP server for quantitative traders assessing whether a backtest result may be caused by testing many variants. It provides overfitting statistics and time-series cross-validation splits that account for leakage.
Key features
- 01CSCV probability of backtest overfitting
- 02Trial-adjusted Deflated Sharpe Ratio
- 03Minimum backtest length calculation
- 04Purged and embargoed CV splits
Use cases
- 01Assess whether a strategy's backtest may be overfit
- 02Adjust Sharpe results for multiple trials
- 03Create time-series validation splits without lookahead leakage
MCP support
Exposes backtest-overfitting statistics and purged cross-validation split generation as tools for MCP clients.
Configuration
Remote servers need no local install. Pick your client, then copy the config or install in one click.
https://overfitting-audit-mcp.mcpize.run/mcpclaude mcp add --transport http tylerscomic-lab-overfitting-audit https://overfitting-audit-mcp.mcpize.run/mcp{"mcpServers":{"tylerscomic-lab-overfitting-audit":{"type":"http","url":"https://overfitting-audit-mcp.mcpize.run/mcp"}}}Open in Cursor{"servers":{"tylerscomic-lab-overfitting-audit":{"type":"http","url":"https://overfitting-audit-mcp.mcpize.run/mcp"}}}Open in VS Code{"mcpServers":{"tylerscomic-lab-overfitting-audit":{"type":"http","url":"https://overfitting-audit-mcp.mcpize.run/mcp"}}}{"mcpServers":{"tylerscomic-lab-overfitting-audit":{"serverUrl":"https://overfitting-audit-mcp.mcpize.run/mcp"}}}Run commands in a terminal; paste JSON into the client’s MCP config and restart. Servers that need sign-in will prompt on first connect.
Tools
Details
From the project READMEoverfitting-audit-mcp
An MCP server that answers "is this edge real, or a testing-hundreds-of-variants artifact?" — implementing the Probability of Backtest Overfitting (CSCV method), Deflated Sharpe Ratio, Minimum Backtest Length, and purged/embargoed cross-validation splits.
The problem this solves
Testing enough parameter combinations against the same historical data will eventually produce a great-looking backtest by chance alone. Standard backtest metrics (Sharpe, win rate, profit factor) don't distinguish a genuine edge from the best-looking result out of hundreds of near-identical variants. This audits for that specific failure mode directly, rather than trusting a single strong-looking curve.
Tools
probability_of_backtest_overfitting
Combinatorially Symmetric Cross-Validation (CSCV) method — estimates the probability that a strategy's in-sample performance rank won't hold out-of-sample.
deflated_sharpe_ratio
Adjusts a Sharpe ratio for the number of trials run and the non-normality of returns, so it can't be inflated just by testing more variants.
minimum_backtest_length
The minimum number of independent trials/observations needed before a given Sharpe ratio is statistically meaningful at all.
purged_cv_split
Generates purged and embargoed cross-validation splits for time-series backtests, preventing the lookahead leakage that ordinary k-fold CV introduces on financial data.
Use it
Hosted (recommended): MCPize — free tier, paid Pro tier for higher limits.
Self-host:
npm install
node server.js
Part of the AlgoForge suite
Prop-firm and quant-validation tools for algo traders: prop-rules-mcp, trade-journal-mcp, payout-calc-mcp, econ-calendar-mcp, montecarlo-validator-mcp, walkforward-validator-mcp, pinescript-audit-mcp, backtest-cost-sensitivity-mcp, pinescript-mcp.
License
MIT
Comments
New to MCP?
From what the protocol is to installing your first server, these answers cover most first questions.
An open protocol that defines how AI applications connect to external tools and data in a uniform way. Clients talk to the model; MCP servers provide tools, resources and prompt templates. They exchange standard messages, so one server can be reused by many clients.