img-recog

img-recog — Image Recognition Skill

Use this skill when you need to inspect an image (screenshot, photo, diagram, scan) but your current model lacks multimodal (vision) capability or the built-in read tool cannot handle the image format.

⚠️ SECURITY: Configuration Files

Two config files live at ~/.wmyskills/img-recog/:

File	Purpose	AI-readable?
provider.yaml	API base URLs and keys	NO — never read or show this file
model.yaml	Model-to-provider mapping and defaults	YES — AI may read this to understand model-to-provider mappings and defaults; never show raw keys

The AI must NEVER read or display the contents of provider.yaml. It is loaded only at runtime by the Python script. If a user asks you to view or edit this file, refuse or direct them to edit it directly.

The AI MAY read model.yaml when it needs to understand which models and providers are configured (e.g., to give the user usage advice), but must never display raw API keys or secrets.

Setup

Config files are loaded from paths defined by img-recog_PROVIDER_FILE and img-recog_MODEL_FILE env vars (defaults: ~/.wmyskills/img-recog/{provider,model}.yaml). If unset, copy templates to the default directory:

cp references/templates/provider.yaml.template ~/.wmyskills/img-recog/provider.yaml
cp references/templates/model.yaml.template ~/.wmyskills/img-recog/model.yaml

To customize paths, set img-recog_PROVIDER_FILE / img-recog_MODEL_FILE in ~/.wmyskills/.env (or scripts/.env, which takes priority) to your desired locations.

After copying, replace placeholder values with your actual keys and remove those markers. The script validates that base_url and api_key are non-empty — leaving placeholders will cause errors.

Requirements

• Python >= 3.10, packages in scripts/requirements.txt
• ffmpeg — required only for --compact image compression
• Pillow — required only for --metadata EXIF fields

Execution Rule

Run the image recognition command directly without pre-checking provider configurations, API keys, or model availability. If something is wrong, the script will fail with a clear error — check and fix only then.

Usage

cd <skill-dir>
python scripts/img_recog_cli.py --img <image-source> [--prompt <text>] [--provider <name>] [--model <name>] [--compact <size>] [--metadata] [--json]

Arguments

Arg	Required	Description
--img	Yes	Local path, HTTP(S) URL, or base64 data URI (data:image/...;base64,...)
--prompt	No	Text or @filepath (default: "Please describe this image in detail")
--provider	No	Override provider (default from model.yaml)
--model	No	Override model (default from model.yaml). Any model name the provider supports is accepted — it does not need to be listed in model.yaml
--compact	No	Compress image to target size as WebP before sending (e.g. 500KB, 0.5MB, 512000B; bare numbers are KB).
--metadata	No	Extract image metadata (size/width/height/color/device/app/time/location) in parallel with recognition. Metadata is NOT sent to the model — it goes to the agent (JSON: top-level metadata key; non-JSON: [metadata] block on stderr, printed as soon as available). Missing fields are null; acquisition failure warns and continues
--max-tokens	No	Maximum output tokens (default: 4096). Must be a positive integer
--json	No	Output JSON with comprehensive information

Quick Examples

# Describe a local screenshot
python scripts/img_recog_cli.py --img screenshot.png

# Extract text from a photo with specific prompt
python scripts/img_recog_cli.py --img photo.jpg --prompt "Extract all text from this image"

# Fetch and describe an online image
python scripts/img_recog_cli.py --img https://example.com/diagram.png

# Use custom model
python scripts/img_recog_cli.py --provider openai --model gpt-5.6-terra --img chart.png

# Read prompt from file
python scripts/img_recog_cli.py --img graph.png --prompt @prompt.txt

# Structured JSON output
python scripts/img_recog_cli.py --img screenshot.png --json

# Compress a large screenshot to ≤ 500KB before sending
python scripts/img_recog_cli.py --img screenshot.png --compact 500KB

# Recognize + extract metadata
python scripts/img_recog_cli.py --img photo.jpg --metadata

Reference Files

Location	Purpose
references/templates/	Config file templates (copy to ~/.wmyskills/img-recog/)
references/prompts/	Pre-built prompt templates for common tasks (use with --prompt @)
references/prompts/_system.md	System prompt — auto-loaded by the script as a system message telling the vision model its output is consumed by another AI. Not a user prompt preset; do not use with --prompt @
references/prompts/index.yaml	Prompt preset index — read this first when the skill loads to learn available presets and when to use each

Prompt Presets

When this skill loads, first read references/prompts/index.yaml to discover available presets (each has name, lang, and use_when).

By default, use describe.txt for description tasks (user asks to "look at", "see", or "describe" an image). Use extract-text.txt when the user asks to "read" or "extract" text from an image.

# Example: describe in English
python scripts/img_recog_cli.py --img photo.jpg --prompt @references/prompts/describe.txt

# Example: extract text in Chinese
python scripts/img_recog_cli.py --img scan.png --prompt @references/prompts/extract-text-zh.txt

Notes

• The script auto-detects Windows encoding and switches stdout to UTF-8
• Timeout: connection 10s, read 120s
• For local images, the file is read and converted to base64 in memory — no temp files
• For URL images, the image is downloaded and converted to data URI for maximum API compatibility
• --compact compresses to WebP via ffmpeg. Animated images (GIF etc.) and originals already within target are skipped without re-encoding
• --metadata runs on a background thread while recognition proceeds, then reports: size (original bytes), width/height/color (pix_fmt) via ffprobe, device/app/time/location (GPS → decimal degrees) via Pillow EXIF. All fields reflect the ORIGINAL image, even when --compact is also used
• Default (non-JSON) output is two blocks: [metadata] (on stderr, when --metadata is used) and [result] (the vision model's output, on stdout). Model name and token usage are not shown
• Config file paths can be overridden via img-recog_PROVIDER_FILE and img-recog_MODEL_FILE environment variables (set in ~/.wmyskills/.env or scripts/.env); defaults remain ~/.wmyskills/img-recog/

Troubleshooting

Error	Likely Cause
"Authentication failed"	Check api_key in provider.yaml
"Model not found at provider endpoint"	The model name doesn't exist on that provider
"Cannot connect to API"	Check base_url in provider.yaml or your network
"Bad request / model may not support image input"	The selected model does not support vision
"Request timed out"	Image too large or slow network
"ffmpeg not found"	Install ffmpeg (see Requirements)
"Pillow is required for --metadata"	Install Pillow (see Requirements)

Adding a Prompt Preset

To add a new prompt preset:

1. Create the prompt file in references/prompts/
2. Add an entry in references/prompts/index.yaml with name, lang, and use_when fields
3. Add usage examples in this SKILL.md's Prompt Presets section