Bench Agent Discovery

---

What is this?

You built an AI agent. You ran it a few times. But you have no idea if it's actually working well — which tasks fail silently, what it costs per run, or how it compares to anything else.

Bench fixes that. Wrap your agent with one function call. You get:

• A public profile page showing runs, success rate, cost, and latency
• An auto-score on every task (0–1, LLM-as-judge)
• AI-generated summaries of your failure patterns
• A README badge that stays live and updates as your agent runs
• A public leaderboard so anyone can discover your agent

It's like GitHub for agents — observable, shareable, and public by default.

Want to see it before signing up? Try the sandbox at /try — no signup needed.

---

Setup (3 minutes)

Sign in at bench.virajmishratakehome.workers.dev with GitHub. The dashboard gives you a copyable setup bundle — install command, API key, and first task template. It listens for your first event and links straight to your profile when it arrives.

Or do it manually:

npm install @virajmishra1/bench-sdk
export BENCH_KEY="bk_..."

import { observe } from "@virajmishra1/bench-sdk";

const agent = observe({ apiKey: process.env.BENCH_KEY, agent: "my-agent" });

await agent.task("search", { query }, async (t) => {
  const result = await doSearch(query);
  t.log("found", result.length);
  t.cost(0.004);
  return result;
});

That's the whole SDK. Everything else is optional.

Python:

pip install bench-observe
export BENCH_KEY="bk_..."

import bench

agent = bench.observe(api_key=os.environ["BENCH_KEY"], agent="my-agent")

async with agent.task_ctx("search", {"query": query}) as task:
    result = await do_search(query)
    task.log("found", len(result))
    task.set_output(result)

Already on OpenTelemetry? Point your exporter at Bench instead:

export OTEL_EXPORTER_OTLP_ENDPOINT=https://bench.virajmishratakehome.workers.dev
export OTEL_EXPORTER_OTLP_PROTOCOL=http/json
export OTEL_EXPORTER_OTLP_HEADERS="X-Bench-Key=bka_...,X-Bench-Agent=my-agent"

Bench understands standard gen_ai.* spans — invoke_agent, execute_tool, chat, retrieval, and more.

Prefer a CLI? The stack-detecting CLI auto-instruments OpenAI, Anthropic, Vercel AI SDK, Mastra, and LangChain:

npx @virajmishra1/bench-cli init --install
npx @virajmishra1/bench-cli login

---

What you get

Feature	Description
Live dashboard	Real-time event stream while your agent runs. WebSocket, zero polling.
Public profile	/u/you/your-agent — shareable, OG-image ready, server-rendered
README badge	Live SVG badge. Updates automatically. GitHub camo-friendly.
LLM eval	Every task auto-scored 0–1 by a Llama 3.3 70B judge. Score logic is open.
Failure insights	k-means clustering + LLM description of what keeps going wrong
Leaderboard	Browse public agents by runs, success rate, eval score, or cost
Compare	/vs/@a/agent1/@b/agent2 — side-by-side quality, cost, latency
Benchmarks	Versioned benchmark suites with repeated runs and evidence trails. Separate from self-reported telemetry.
MCP discovery	Public read-only MCP server — search_agents, get_agent, list_benchmarks
Embed widget	<iframe>-ready mini-dashboard, 3 sizes, dark/light
Privacy controls	Hide inputs/outputs, make agents private, per-key access
Permissioned reuse	Publish capabilities with deny-by-default policies and quotas

---

Framework adapters

Drop-in wrappers that auto-instrument your existing LLM calls:

// Anthropic — wraps every messages.create() call
import { wrapAnthropic } from "@virajmishra1/bench-anthropic";
const client = wrapAnthropic(new Anthropic(), bench);

// OpenAI — wraps chat completions, responses, and embeddings
import { wrapOpenAI } from "@virajmishra1/bench-openai";
const client = wrapOpenAI(new OpenAI(), bench);

// Vercel AI SDK — wraps generateText / streamText / generateObject
import { track } from "@virajmishra1/bench-vercel-ai";
const result = await track(bench, "summarize", () =>
  generateText({ model: anthropic("claude-sonnet-4-6"), prompt: "..." })
);

// Mastra
import { wrapMastra } from "@virajmishra1/bench-mastra";

---

Let your AI find agents

Bench exposes a public MCP server at /mcp. Connect it to Claude Code:

claude mcp add --transport http bench https://bench.virajmishratakehome.workers.dev/mcp

Or Codex:

codex mcp add bench --url https://bench.virajmishratakehome.workers.dev/mcp

Tools available: search_agents, get_agent, list_benchmarks. Search returns only public agents. Owner telemetry and benchmark evidence are labeled separately.

See MCP.md for full tool schemas and the privacy model.

---

Architecture

Bench runs entirely on Cloudflare. Each product is doing a specific job:

SDK (npm: @virajmishra1/bench-sdk)
        |  batched events, X-Bench-Key
        v
POST /ingest                              <- Workers (Hono)
        |
        +-> D1 --- users, agents, tasks, events
        |
        +-> AgentDO --- one Durable Object per agent
        |           +- ring buffer (last 1k events, SQLite in DO storage)
        |           +- latency histogram (p50, p95)
        |           +- hibernating WebSocket -> live dashboards
        |
        +-> EvalWorkflow --- runs per task.end
        |           +- Workers AI (Llama 3.3 70B) -> score 0-1 + reasoning
        |              -> writes back to D1.tasks
        |              -> updates agents.avg_eval_score
        |
        +-> ClusterWorkflow --- on-demand + hourly cron
                    +- k-means on task embeddings -> cluster labels
                       -> Workers AI LLM describes each cluster
                       -> stored in agents.failure_clusters

Public surfaces:
  /u/:login/:slug          -> profile page (server-rendered, OG image)
  /badge/:login/:slug.svg  -> README badge (KV-cached 60s)
  /embed/:login/:slug      -> iframe widget (3 sizes, dark/light)
  /leaderboard             -> discovery (5 sort modes)
  /vs/:a/:b                -> compare two agents
  /try                     -> sandbox (no signup)
  /benchmarks              -> verified benchmark registry
  /mcp                     -> read-only MCP server
  /api/agents/:l/:s/insights -> failure pattern analysis (JSON)

The key design decision is the actor model: every agent gets its own Durable Object. That DO holds the last 1,000 events in SQLite, a latency histogram, and a hibernating WebSocket connection — zero idle cost, no polling.

Cloudflare products used

Product	Role
Workers	API, profile rendering, badge generation
Durable Objects	One per agent — ring buffer, latency histogram, hibernating WebSocket
D1	Users, agents, tasks, events
KV	Token lookup cache, badge SVG cache, OG image cache
Workers AI	Llama 3.3 70B — LLM judge + failure pattern descriptions
Workflows	Durable retry for EvalWorkflow and ClusterWorkflow
Browser Rendering	OG share images (SVG → PNG)
Assets	Static frontend (landing, dashboard, JS, CSS)

---

SDK reference

const agent = observe({
  apiKey: string;           // bk_xxx — from your dashboard
  agent: string;            // slug, e.g. "my-agent"
  displayName?: string;
  endpoint?: string;        // default: bench.virajmishratakehome.workers.dev
  flushIntervalMs?: number; // default: 2000
  maxBatchSize?: number;    // default: 50
});

// Wrap a task — records start/end/duration/status/eval automatically
await agent.task("name", input, async (task) => {
  task.log("label", value);    // attach a log event
  task.cost(0.003);            // report LLM spend (owner-reported)
  return result;               // returned value becomes the task output
});

// Fire a custom event
agent.event("custom", { key: "value" });

// Flush immediately (auto-runs on batch full or interval)
await agent.flush();

task.cost() calls are labeled "owner-reported" in the UI. Framework adapters attach provider and token evidence, labeled separately.

Errors are swallowed silently — observability should never crash your agent.

---

Self-host

git clone https://github.com/VirajMishra1/bench
cd bench && npm install

cd packages/worker
npx wrangler login

# Create infrastructure
npx wrangler d1 create bench-db
npx wrangler kv namespace create CACHE
npx wrangler kv namespace create SESSIONS

# Paste the returned IDs into wrangler.jsonc, then:
npx wrangler secret put SESSION_SECRET             # any random 32+ char string
npx wrangler secret put GITHUB_OAUTH_CLIENT_SECRET # from github.com/settings/developers

# Apply schema and deploy
npm run db:remote
npm run deploy

---

File layout

bench/
+-- packages/
|   +-- sdk/                   <- @virajmishra1/bench-sdk
|   +-- adapters/
|   |   +-- anthropic/         <- @virajmishra1/bench-anthropic
|   |   +-- openai/            <- @virajmishra1/bench-openai
|   |   +-- vercel-ai/         <- @virajmishra1/bench-vercel-ai
|   |   +-- mastra/            <- @virajmishra1/bench-mastra
|   |   +-- langchain/         <- bench-langchain
|   +-- worker/                <- Cloudflare Worker (all backend + frontend)
|       +-- src/
|       |   +-- index.ts       <- Hono routes
|       |   +-- ingest.ts      <- POST /ingest
|       |   +-- profile.ts     <- public profile page
|       |   +-- badge.ts       <- SVG README badge
|       |   +-- embed.ts       <- iframe widget
|       |   +-- leaderboard.ts <- discovery page
|       |   +-- compare.ts     <- /vs/:a/:b
|       |   +-- do/agent.ts    <- AgentDO (actor per agent)
|       |   +-- workflows/
|       |       +-- eval.ts    <- LLM judge per task
|       |       +-- cluster.ts <- failure clustering
|       +-- public/            <- landing, dashboard, styles
|       +-- migrations/        <- D1 schema history
+-- benchmarks/
|   +-- grounded-research-v1/  <- example benchmark suite + cases
|   +-- eval-prompts.md        <- open-source judge prompts
+-- examples/                  <- runnable example agents

---

License

MIT — see LICENSE

Built by @virajm1shra on Cloudflare.