mcpField
F
flutter-mcp-toolkit-dogfood-iterations

flutter-mcp-toolkit-dogfood-iterations

github.com/arenukvernTesting & quality39448 installsUpdated 2w ago
Repository

About flutter-mcp-toolkit-dogfood-iterations

A dogfood iteration skill for flutter-mcp-toolkit developers. It describes runtime and static evaluation workflows, score tracking, and how to turn repeated warnings into actionable fixes.

Key features

  1. 01Run the web evaluation workflow
  2. 02Run static evaluation gates
  3. 03Record scores and warnings
  4. 04Compare scores by dimension
  5. 05Track recurring issues and fixes

Use cases

  1. 01Check runtime behavior in the web showcase
  2. 02Run static checks in CI or PRs
  3. 03Track quality changes across evaluation runs

MCP support

It evaluates MCP tool calls, connectivity, and WebMCP parity, and documents the naming differences between exec and MCP.

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/arenukvern/mcp_flutter --skill flutter-mcp-toolkit-dogfood-iterations

After installing, type/flutter-mcp-toolkit-dogfood-iterationsin Claude Code to confirm it loaded.

Details

From the project README

Dogfood iterations

Tracker: docs/evidence/dogfood/dogfood_web_eval.yaml
Rubric: docs/superpowers/evals/tool_quality_rubric.yaml
Overview: docs/superpowers/evals/README.md

Pass threshold

≥80/100 weighted; ship at ≥90 or accepted pass_with_warnings.

DimensionWeight
connectivity20
schema_validity10
handler_correctness20
capture_quality10
visual_fidelity10
webmcp_parity10
intentcall_authoring10
docs_truth10

Iteration workflow

Task Progress:
- [ ] Start app (web-showcase or make showcase)
- [ ] Fresh WS_URI from log (not stale after hot reload)
- [ ] Run battery
- [ ] Record score + warnings in tracker
- [ ] Promote recurring patterns if repeated ≥2 times
Web iteration
make web-showcase
export WS_URI="$(grep -Eo 'ws://127\.0\.0\.1:[0-9]+/[A-Za-z0-9_=-]+/ws' .showcase/web_app.log | tail -1)"
bash tool/evals/run_dogfood_eval.sh --ws-uri "$WS_URI" --merge --skip-visual   # yq or dart fallback
# or without merge:
bash tool/evals/run_dogfood_eval.sh --ws-uri "$WS_URI"

Auto-runs: static gates + validate-runtime + webmcp verify.

Static only (CI / PR)
make dogfood-eval-static
Makefile
make dogfood-eval          # needs WS_URI in env
make dogfood-eval-static

Artifacts

  • Per run: .showcase/eval_runs/<timestamp>/eval_run.yaml
  • Snapshot: .showcase/eval_run_<timestamp>.yaml
  • With --merge: updates docs/evidence/dogfood/dogfood_web_eval.yaml (yq or dart run mcp_server_dart/tool/merge_dogfood_tracker.dart)

Verdicts

VerdictMeaning
passscore ≥80, no blocking warnings
pass_with_warningsscore ≥80, e.g. visual_capture_truth_mode
failscore <80
blocked_no_runtime--skip-runtime only

Exec vs MCP names

Document in iteration notes when testing invoke:

  • exec: bare names (get_recent_logs, dogfood_ping) — also accepts fmt_* aliases
  • MCP: fmt_ prefix (fmt_get_recent_logs, fmt_client_tool + listing name)
  • Connection: global --vm-service-uri (not exec --target)
dart run mcp_server_dart/bin/flutter_mcp_toolkit.dart \
  --vm-service-uri "$WS_URI" exec --name get_recent_logs --args '{}'

Chrome battery notes

  • Skip the visual track (--skip-visual): its fixture home (external flutter_harness) is retired, so the visual evidence is paused, not regenerable
  • validate-runtime --save-images can hang >5m on Chrome; battery omits it unless DOGFOOD_SAVE_IMAGES=1

CI (branch)

.github/workflows/intentcall_eval.yml — PR: dogfood-eval-static; weekly/main: intentcall package tests + static.

Full Chrome runtime dogfood stays local until headless WebMCP is cost-effective.

After each iteration

  1. Diff dimension_scores vs previous iteration in tracker.
  2. Add to recurring_warnings / recurring_errors when repeated.
  3. Update fix_recommendations with actionable doc/code deltas.
  4. Do not treat tracker YAML as gate without a fresh battery run.

Related skills

  • flutter-mcp-toolkit-maintain-web — WebMCP launch + verify
  • flutter-mcp-toolkit-maintain-macos — macOS showcase
  • flutter-mcp-toolkit-repo-maintainer — make sync-skills after editing this skill

Comments

Markdown supported