mcpField
W
web-search

web-search

github.com/yejinleiProductivity2374 installsUpdated 34w ago
Repository

About web-search

web-search is a script-based skill for searching the web, exploring results in depth, and crawling pages. It uses several search engines with automatic fallback and returns structured results without requiring API keys.

Key features

  1. 01Search across multiple engines
  2. 02Automatically fall back when an engine fails
  3. 03Return titles, URLs, and snippets
  4. 04Run deep searches and extract content
  5. 05Crawl a specified web page

Use cases

  1. 01Find information across search engines
  2. 02Research a topic and inspect page content
  3. 03Extract content from a known URL

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/yejinlei/web-search-skill --skill web-search

After installing, type/web-searchin Claude Code to confirm it loaded.

Bundled files

References and scripts installed along with SKILL.md.

LICENSE.txt
.gitignore
requirements.txt
MANIFEST.in
README.md
setup.py
scripts/
web_search.py
__init__.py

Details

From the project README

Web Search Skill

A powerful web search skill supporting multiple search engines without requiring API keys.

Features

  • 🔍 Multi-Engine Support: Baidu (Playwright), Bing, DuckDuckGo
  • 🌐 No API Key Required: Uses browser automation and web scraping
  • 🔄 Smart Fallback: Automatically switches engines when one fails
  • 📊 Structured Results: Returns clean search results with title, URL, and snippet
  • 🚀 High Performance: Async support with Playwright browser automation

Usage

Basic Search
result = main({
    "action": "search",
    "query": "Python tutorial",
    "num_results": 5
})
Deep Search
result = main({
    "action": "deep_search",
    "query": "machine learning latest research",
    "num_results": 5
})
Web Page Crawling
result = main({
    "action": "crawl",
    "url": "https://example.com"
})

Input Parameters

ParameterTypeRequiredDescription
actionstringYesOperation type: "search", "deep_search", or "crawl"
querystringConditionalSearch query (required for search/deep_search)
urlstringConditionalTarget URL (required for crawl)
num_resultsintNoNumber of results, default 5, max 20
regionstringNoRegion code, default 'cn-zh'

Output Format

Search Result
{
    "success": True,
    "query": "search query",
    "engine": "baidu+playwright",
    "num_results": 5,
    "results": [
        {
            "title": "Result title",
            "href": "https://...",
            "body": "Snippet content"
        }
    ],
    "message": "Search completed"
}
Deep Search Result
{
    "success": True,
    "query": "search query",
    "search_results": [...],
    "detailed_info": {
        "extracted_content": "..."
    },
    "message": "Deep search completed"
}

Execution

type: script script_path: scripts/web_search.py entry_point: main dependencies:

  • uv>=0.1.0
  • requests>=2.28.0
  • baidusearch>=1.0.3
  • crawl4ai>=0.8.0
  • playwright>=1.40.0

Search Strategy

  1. Primary: baidusearch library (fastest, no browser)
  2. Secondary: Playwright + Baidu (most reliable, bypasses anti-bot)
  3. Tertiary: DuckDuckGo (privacy-focused)
  4. Fallback: Bing (international)

Notes

  1. First Run: Playwright will download Chromium browser on first use (~100MB)
  2. Rate Limiting: Be mindful of search frequency to avoid temporary blocks
  3. Network: Requires internet connection
  4. Results: May vary based on search engine algorithms and location

Error Handling

  • Returns {"success": False, "message": "..."} on errors
  • Automatically retries with fallback engines
  • Graceful degradation when optional dependencies are missing

Comments

Markdown supported