mistral-ocr

Mistral OCR

Extract text from images and PDFs using Mistral's dedicated OCR API. No external dependencies required.

Requirements

This skill requires a Mistral API key. If you don't have one, follow the guide in reference/getting-started.md.

API Key

The user must provide their Mistral API key. Ask for it if not available.

Option 1 (Recommended for AI agents): User provides key directly in message:

"Use this Mistral key: aBc123XyZ..."
"Convert this PDF to markdown, my API key is aBc123XyZ..."

Option 2: Environment variable $MISTRAL_API_KEY

Option 3: Claude Code settings (~/.claude/settings.json)

If no key is available, guide the user to get one at console.mistral.ai.

---

API Endpoint

Use the dedicated OCR endpoint for all document processing:

POST https://api.mistral.ai/v1/ocr

Model: mistral-ocr-latest

---

Features

1. PDF → Markdown (Direct, no conversion needed!)

curl -s "https://api.mistral.ai/v1/ocr" \
  -H "Authorization: Bearer $MISTRAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistral-ocr-latest",
    "document": {
      "type": "document_url",
      "document_url": "https://example.com/document.pdf"
    }
  }'

2. Image → Text

Works with JPG, PNG, WEBP, GIF:

curl -s "https://api.mistral.ai/v1/ocr" \
  -H "Authorization: Bearer $MISTRAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistral-ocr-latest",
    "document": {
      "type": "image_url",
      "image_url": "https://example.com/image.jpg"
    }
  }'

3. Local Files (Base64 Data URL)

For local PDFs or images, encode as base64 and use a data URL.

ALWAYS use curl (works on all platforms including Windows via Git Bash):

# For local PDF
BASE64=$(base64 -w0 document.pdf)
curl -s "https://api.mistral.ai/v1/ocr" \
  -H "Authorization: Bearer $MISTRAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistral-ocr-latest",
    "document": {
      "type": "document_url",
      "document_url": "data:application/pdf;base64,'"$BASE64"'"
    }
  }'

# For local images (PNG, JPG, etc.)
BASE64=$(base64 -w0 image.png)
curl -s "https://api.mistral.ai/v1/ocr" \
  -H "Authorization: Bearer $MISTRAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistral-ocr-latest",
    "document": {
      "type": "image_url",
      "image_url": "data:image/png;base64,'"$BASE64"'"
    }
  }'

MIME types:

• PDF: data:application/pdf;base64,...
• PNG: data:image/png;base64,...
• JPG: data:image/jpeg;base64,...
• WEBP: data:image/webp;base64,...

4. Structured JSON Output

For invoices, forms, tables - ask for JSON in a follow-up or use Document AI annotations.

---

Response Format

The API returns markdown directly:

{
  "pages": [
    {
      "index": 0,
      "markdown": "# Document Title\n\nExtracted content here...",
      "images": [],
      "tables": [],
      "dimensions": {"dpi": 200, "height": 842, "width": 595}
    }
  ],
  "model": "mistral-ocr-latest",
  "usage_info": {"pages_processed": 1, "doc_size_bytes": 12345}
}

---

Workflow

User requests OCR from image or PDF

1. Get API key - Ask user if not in environment
2. Determine input type (URL or local file)
3. For local files, ALWAYS use temp file approach (avoids "Argument list too long" error):

# Cross-platform temp directory
TMPDIR="${TMPDIR:-${TEMP:-/tmp}}"

# Step 1: Encode file to base64
base64 -w0 "document.pdf" > "$TMPDIR/b64.txt"

# Step 2: Create JSON request file
echo '{"model":"mistral-ocr-latest","document":{"type":"document_url","document_url":"data:application/pdf;base64,'$(cat "$TMPDIR/b64.txt")'"}}' > "$TMPDIR/request.json"

# Step 3: Call API with -d @file (use actual key, not variable)
curl -s "https://api.mistral.ai/v1/ocr" \
  -H "Authorization: Bearer YOUR_API_KEY_HERE" \
  -H "Content-Type: application/json" \
  -d @"$TMPDIR/request.json" > "$TMPDIR/response.json"

# Step 4: Extract markdown with node (NOT jq - not available on all systems)
node -e "const fs=require('fs'); const r=JSON.parse(fs.readFileSync('$TMPDIR/response.json')); console.log(r.pages.map(p=>p.markdown).join('\n\n---\n\n'))"

4. Save to .md file using Write tool
5. Confirm file location to user

IMPORTANT: Cross-Platform Compatibility

• ALWAYS use curl (works on Windows via Git Bash)
• ALWAYS use -d @file for request body (handles large files)
• NEVER use jq - use node instead to parse JSON
• Use ${TMPDIR:-${TEMP:-/tmp}} for temp files (works on all systems)
• Copy response.json to user directory before parsing with node on Windows

---

Usage Examples

When the user says:

User Request	Action
"Convert this PDF to markdown"	OCR the PDF, save as .md file
"Extract text from this image"	OCR the image, return text
"Give me a .md of this document"	OCR and save as .md file
"What does this PDF say?"	OCR and summarize content
"OCR this receipt"	Extract text, optionally structure as JSON

---

Error Handling

Error	Cause	Solution
401 Unauthorized	Invalid API key	Verify key, guide to getting-started.md
400 Bad Request	Invalid document	Check format and URL accessibility
3310 File fetch error	URL not accessible	Use base64 for local files
Rate limit	Too many requests	Wait and retry

---

Supported Formats

Format	Support
PDF	✅ Direct (no conversion)
PNG	✅ Direct
JPG/JPEG	✅ Direct
WEBP	✅ Direct
GIF	✅ Direct

No external dependencies required! Unlike other OCR solutions, Mistral OCR handles PDFs directly without needing pdftoppm, ImageMagick, or any other tools.

---

Pricing

As of 2025, Mistral OCR pricing:

• $2 per 1,000 pages
• 50% discount with Batch API

Check current rates at mistral.ai/pricing

---

References

• Getting Started - How to get your API key
• PDF to Markdown - PDF conversion examples
• Output Formats - JSON, Markdown, plain text
• Step-by-Step Guide - Complete tutorial with examples

---

Skill by Parlamento AI