mcpField
S
smolvlm

smolvlm

github.com/tdiminoCreative & media4155 installsUpdated 1w ago
Repository

About smolvlm

smolvlm is a skill for analyzing images locally with SmolVLM-2B on Apple Silicon. It supports image descriptions, visual questions, OCR, and interface or code screenshot analysis through a Python script.

Key features

  1. 01Local image analysis with SmolVLM-2B
  2. 02Image descriptions and visual questions
  3. 03OCR and screenshot text extraction
  4. 04UI and code screenshot analysis
  5. 05PNG, JPG, GIF, and WebP support

Use cases

  1. 01Describe a photo stored locally
  2. 02Extract text from a screenshot
  3. 03Inspect interface controls and state
  4. 04Identify code or errors in an image

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/tdimino/claude-code-minoan --skill smolvlm

After installing, type/smolvlmin Claude Code to confirm it loaded.

Bundled filesREADME.mdscripts/view_image.py

Details

From the project README

SmolVLM - Local Image Analysis

Analyze images locally using SmolVLM-2B, a state-of-the-art compact vision-language model optimized for Apple Silicon via mlx-vlm.

Quick Usage

Describe an Image
python ~/.claude/skills/smolvlm/scripts/view_image.py /path/to/image.png
Ask a Question About an Image
python ~/.claude/skills/smolvlm/scripts/view_image.py /path/to/image.png "What text is visible?"
Specific Tasks
# Extract text (OCR)
python ~/.claude/skills/smolvlm/scripts/view_image.py screenshot.png "Extract all text"

# UI analysis
python ~/.claude/skills/smolvlm/scripts/view_image.py ui.png "Describe the UI elements"

# Detailed description
python ~/.claude/skills/smolvlm/scripts/view_image.py photo.jpg --detailed

Effective Prompts

General Description
  • "Describe this image" - Basic description
  • "Describe this image in detail, including colors, composition, and any text" - Comprehensive
Text Extraction (OCR)
  • "Extract all visible text from this image"
  • "What text appears in this screenshot?"
  • "Read the text in this document"
UI/Screenshot Analysis
  • "Describe the user interface elements"
  • "What buttons and controls are visible?"
  • "Identify the application and its current state"
Visual Question Answering
  • "How many [objects] are in this image?"
  • "What color is the [object]?"
  • "Is there a [object] in this image?"
Code/Technical
  • "What programming language is shown?"
  • "Describe what this code does"
  • "Identify any errors in this code screenshot"

Model Details

SpecValue
ModelSmolVLM-2B-Instruct
Size~4GB
Peak Memory5.8GB
Speed~94 tok/s (M-series)
Supported FormatsPNG, JPG, JPEG, GIF, WebP

Requirements

  • macOS with Apple Silicon (M1/M2/M3)
  • Python 3.10+
  • mlx-vlm package: uv pip install mlx-vlm --system

Troubleshooting

"Model not found": First run downloads the model (~4GB). Wait for completion.

Out of memory: Close other applications. Model needs ~6GB free RAM.

Slow first inference: Model loading takes 10-15s on first use, subsequent calls are faster.

Comments

Markdown supported