mcpField
G
gs-search

gs-search

github.com/cookjohnProductivity538443 installsUpdated 33w ago
Repository

About gs-search

A skill for searching Google Scholar by keyword and presenting scraped results in a structured list. It includes CAPTCHA handling and follow-up steps for paging, citation lookup, and Zotero export through related skills.

Key features

  1. 01Keyword search on Google Scholar
  2. 02Structured paper result extraction
  3. 03Displays data-CID identifiers
  4. 04Highlights available full text
  5. 05CAPTCHA detection and retry guidance

Use cases

  1. 01Finding papers on a research topic
  2. 02Checking citation counts and paper identifiers
  3. 03Preparing results for follow-up export to Zotero

MCP support

Uses the mcp__chrome-devtools__navigate_page tool to open Google Scholar and evaluate_script to scrape result data from the page DOM. It relies on browser tools rather than a public Google Scholar API.

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/cookjohn/gs-skills --skill gs-search

After installing, type/gs-searchin Claude Code to confirm it loaded.

Details

From the project README

Google Scholar Basic Search

Search Google Scholar for papers using keyword(s). Returns structured result list via DOM scraping.

Arguments

$ARGUMENTS contains the search keyword(s).

Steps

1. Navigate

Use mcp__chrome-devtools__navigate_page:

  • url: https://scholar.google.com/scholar?q={URL_ENCODED_KEYWORDS}&hl=en&num=10
2. Extract results (evaluate_script)

Wait for results to load, check for CAPTCHA, then scrape the DOM:

async () => {
  // Wait for results or CAPTCHA
  for (let i = 0; i < 20; i++) {
    if (document.querySelector('#gs_res_ccl') || document.querySelector('#gs_captcha_ccl')) break;
    await new Promise(r => setTimeout(r, 500));
  }

  // CAPTCHA check
  if (document.querySelector('#gs_captcha_ccl') || document.body.innerText.includes('unusual traffic')) {
    return { error: 'captcha', message: 'Google Scholar requires CAPTCHA verification. Please complete it in your browser, then tell me to continue.' };
  }

  const items = document.querySelectorAll('#gs_res_ccl .gs_r.gs_or.gs_scl');
  const results = Array.from(items).map((item, i) => {
    const titleEl = item.querySelector('.gs_rt a');
    const meta = item.querySelector('.gs_a')?.textContent || '';
    // Parse "Author1, Author2 - Journal, Year - publisher"
    const parts = meta.split(' - ');
    const authors = parts[0]?.trim() || '';
    const journalYear = parts[1]?.trim() || '';
    const citedByEl = item.querySelector('.gs_fl a[href*="cites"]');
    const relatedEl = item.querySelector('.gs_fl a[href*="related"]');
    const versionsEl = item.querySelector('.gs_fl a[href*="cluster"]');

    return {
      n: i + 1,
      title: titleEl?.textContent?.trim() || item.querySelector('.gs_rt')?.textContent?.trim() || '',
      href: titleEl?.href || '',
      authors,
      journalYear,
      citedBy: citedByEl?.textContent?.match(/\d+/)?.[0] || '0',
      citedByUrl: citedByEl?.href || '',
      dataCid: item.getAttribute('data-cid') || '',
      fullTextUrl: (item.querySelector('.gs_ggs a') || item.querySelector('.gs_or_ggsm a'))?.href || '',
      snippet: item.querySelector('.gs_rs')?.textContent?.trim()?.substring(0, 200) || '',
      relatedUrl: relatedEl?.href || '',
      versionsUrl: versionsEl?.href || '',
      versions: versionsEl?.textContent?.match(/\d+/)?.[0] || ''
    };
  });

  const totalText = document.querySelector('#gs_ab_md')?.textContent?.trim() || '';
  const currentUrl = window.location.href;
  return { total: totalText, resultCount: results.length, currentUrl, results };
}
3. Report

Present results as a numbered list:

Searched Google Scholar for "$ARGUMENTS": {total}

1. {title}
   Authors: {authors} | {journalYear}
   Cited by: {citedBy} | [Full text]({fullTextUrl})
   Data-CID: {dataCid}

2. ...

Always show the dataCid — it's the unique identifier used for citation export and "cited by" tracking.

If fullTextUrl is available, highlight it (means open-access PDF/HTML).

4. Follow-up

When the user wants to:

  • See more results: use gs-navigate-pages to go to next page
  • See who cited a paper: use gs-cited-by with the data-cid
  • Export to Zotero: use gs-export with the data-cid(s)

CAPTCHA Handling

If the result contains {error: 'captcha'}:

  1. Tell the user: "Google Scholar is requesting CAPTCHA verification. Please complete it in your browser."
  2. Wait for user confirmation
  3. Retry the evaluate_script extraction

Notes

  • This skill uses 2 tool calls: navigate_page + evaluate_script
  • Google Scholar has NO public API — all data extraction is via DOM scraping
  • data-cid is the primary identifier (cluster ID) — used across all GS skills
  • Keep request frequency low to avoid triggering CAPTCHA
  • Default num=10 results per page (max 20)

Comments

Markdown supported