About Calibration Explorer
Calibration Explorer is a local MCP server for assessing whether classifier confidence matches accuracy and generating project-local reports. It fits temperature scaling and evaluates an acceptance threshold without training or running a classifier.
Key features
- 01Load selected prediction files
- 02Assess confidence calibration
- 03Fit temperature scaling
- 04Evaluate and lock an acceptance threshold
- 05Create HTML and JSON reports
Use cases
- 01Check confidence against classifier accuracy
- 02Calibrate predictions using calibration rows
- 03Set an acceptance threshold on validation rows
- 04Save assessment reports in a project
MCP support
The local stdio MCP server exposes tools for loading and assessing predictions, fitting temperature, setting policy, reviewing test outcomes, and creating reports.
Configuration
Local server config is the same for every client. Copy it into your client’s MCP config file.
Tools
Local servers expose tools only once running on your machine, and the maintainer has not provided a tool list.
Details
From the project READMECalibration Explorer MCP
Local stdio MCP server for Calibration Explorer. It checks whether a classifier's saved confidence matches how often it was correct, fits one temperature on calibration rows, evaluates and locks an acceptance threshold on policy-validation rows, and writes HTML and JSON assessment reports into your project.
It reads prediction files you select, uses the same parser, numerical library, workflow guards and report generator as the browser app, and does not train or run a classifier. There is no telemetry or remote service. Returned summaries can still reach your MCP client's model provider.
Use with Claude Code
Save this configuration, replacing both directories with absolute paths that already exist:
{
"mcpServers": {
"calibration": {
"type": "stdio",
"command": "npx",
"args": [
"-y", "@m-sanchez/[email protected]",
"--input-root", "ABSOLUTE_PREDICTION_DIRECTORY",
"--output-root", "ABSOLUTE_REPORT_DIRECTORY"
]
}
}
}
Then launch claude --mcp-config calibration.mcp.json --strict-mcp-config and check /mcp. The verified client is the Claude Code interactive terminal, tested against the server run from a checkout; this npm route and the plugin have passed the clean-install test suite but not yet a full session in Claude Code. Other MCP hosts are not claimed as verified. Node.js 20 or newer is required.
--input-root and --output-root are required and repeatable. Paths outside them are refused, reports go into new folders only, and existing files are never overwritten. --help and --version print and exit.
Tools
load_predictions, assess_predictions, fit_temperature, set_policy, review_test and create_report. Test outcomes stay unopened until review_test receives an explicit decision through the client's elicitation form. Input formats, the complete workflow, disclosure rules and limits are documented in docs/mcp.md.
What the package contains
dist/cli.js is a readable, unminified bundle of the server, the shared app modules and @m-sanchez/calibrated from the exact Git commit the app pins. The official MCP TypeScript SDK and zod are exact-pinned runtime dependencies. The UCI digits reference ships under public/examples with its CC BY 4.0 attribution. Reports record the same app source and numerical distribution hashes as a checkout of the same commit; dist/NOTICE.txt lists them with the bundled licence.
MIT licence. Copyright (c) 2026 Miguel Sánchez Durán.
Comments
New to MCP?
From what the protocol is to installing your first server, these answers cover most first questions.
An open protocol that defines how AI applications connect to external tools and data in a uniform way. Clients talk to the model; MCP servers provide tools, resources and prompt templates. They exchange standard messages, so one server can be reused by many clients.