mcpField
W
Woodstitch

Woodstitch

Local
woodstitch.comMedia & Design0Updated 1w ago
RepositoryVisit website

About Woodstitch

Woodstitch is an MCP server for evaluating text-to-speech audio through paid, blinded human-listener studies. It supports pairwise preference and human-versus-AI detection, with scores and confidence intervals for agents.

Key features

  1. 01Compare two clips by listener preference
  2. 02Detect human or AI audio
  3. 03Upload audio files or use public URLs
  4. 04Quote study costs before starting
  5. 05Retrieve results and export judgments

Use cases

  1. 01Compare candidate TTS systems on the same text
  2. 02Check whether a clip sounds human- or AI-generated
  3. 03Review study costs before authorizing it

MCP support

The MCP server exposes tools to upload audio, create and price studies, start or cancel evaluations, check progress, retrieve results, and export judgments. Starting a study requires a confirmed amount, which sets a spending ceiling.

Configuration

Local server config is the same for every client. Copy it into your client’s MCP config file.

mcp.json · Universal config
{
  "mcpServers": {
    "com-woodstitch-tts-human-eval": {
      "command": "uvx",
      "args": [
        "woodstitch-mcp"
      ],
      "env": {
        "WOODSTITCH_API_KEY": "<WOODSTITCH_API_KEY>"
      }
    }
  }
}
Run commanduvx woodstitch-mcp
Environment variables to set
WOODSTITCH_API_KEYSecretWoodstitch API key (ws_live_...), from https://woodstitch.com/app/api-keys. Not needed for the templates and pricing tools.

Local servers need a runtime on your machine (e.g. Node.js 18+). Config file locations differ by client; see your client’s docs.

Packages
pypiwoodstitch-mcp0.1.5

Tools

13

Saved tool list · updated 6d ago

This tool takes no parameters.

Details

From the project README

woodstitch-mcp

An MCP server for the Woodstitch API: paid human listeners judge your text-to-speech audio in blinded studies, and your agent gets a number back with a confidence interval.

Two tasks, and only two: pairwise preference (a listener hears two clips of the same text and picks one, forced choice) and human-vs-AI detection (one clip, human or AI, confidence 1–4). Not offered: MOS, CMOS, MUSHRA or any 1–5 scale.

Install

Get an API key at https://woodstitch.com/app/api-keys, then:

Claude Code

claude mcp add woodstitch -e WOODSTITCH_API_KEY=ws_live_... -- uvx woodstitch-mcp

Claude Desktop / Cursor / any MCP client (mcpServers entry):

{
  "woodstitch": {
    "command": "uvx",
    "args": ["woodstitch-mcp"],
    "env": {"WOODSTITCH_API_KEY": "ws_live_..."}
  }
}

pipx run woodstitch-mcp or pip install woodstitch-mcp && woodstitch-mcp work too. Requires Python 3.10+.

Claude Code users: the Woodstitch plugin bundles this server with a skill that knows when and how to run a listening test: /plugin marketplace add woodstitch-llc/claude-plugin then /plugin install tts-human-eval@woodstitch.

Tools

ToolWhat it doesMoney
woodstitch_templatesWhat is measured and what is not (no key needed)–
woodstitch_pricingRate card and worked examples (no key needed)–
woodstitch_accountBalance and spend–
woodstitch_upload_filesUpload local audio, get file ids–
woodstitch_create_evalCreate and price a study; dry_run to price onlycharges nothing
woodstitch_start_evalStart the studyspends money
woodstitch_get_evalProgress and provisional score–
woodstitch_get_resultsFinal numbers–
woodstitch_exportEvery judgement as rows (json or csv, optionally to a file)–
woodstitch_cancel_evalStop early, pay only for work done–
woodstitch_list_evals, woodstitch_list_files, woodstitch_delete_filehousekeeping–

Safeguards built in

  • woodstitch_start_eval requires user_confirmed_amount_usd, the quoted total the user agreed to, and sends it to the API as the hard ceiling max_total_usd. If the quote is higher than what the user confirmed, the API refuses (over_max) and nothing starts. An agent cannot start a study the user has not seen the price of.
  • Every POST carries an Idempotency-Key, so a retried call never creates or starts a study twice.
  • The server's instructions tell the model that interim numbers are provisional and that stopping early on a threshold is optional stopping.

Typical session

  1. woodstitch_upload_files(["a/001.wav", "b/001.wav", ...]) or use public URLs.
  2. woodstitch_create_eval(pairs=[{"id":"001","a":"file_…","b":"file_…","passage":"001","system_a":"base","system_b":"candidate"}, …], ratings_per_pair=5) → quote, nothing charged.
  3. Show the user the total. On yes: woodstitch_start_eval(eval_id, user_confirmed_amount_usd=<total>).
  4. woodstitch_get_eval(eval_id) now and then; first judgements usually land within minutes.
  5. woodstitch_get_results(eval_id) when status is completed; woodstitch_export for the rows.

Method, listener pay and controls: https://woodstitch.com/research/methodology. Machine-readable API spec: https://woodstitch.com/openapi.json.

Development

uv venv && . .venv/bin/activate && uv pip install -e .
WOODSTITCH_API_KEY=ws_live_... woodstitch-mcp        # stdio server

MIT © Woodstitch LLC

Comments

Markdown supported

New to MCP?

From what the protocol is to installing your first server, these answers cover most first questions.

An open protocol that defines how AI applications connect to external tools and data in a uniform way. Clients talk to the model; MCP servers provide tools, resources and prompt templates. They exchange standard messages, so one server can be reused by many clients.