mcpField
C
conference-speaker-scraper

conference-speaker-scraper

github.com/gooseworks-aiMarketing & growth1.3k57 installsUpdated 3d ago
Repository

About conference-speaker-scraper

A command-line skill that extracts speaker details from conference pages for event research and outreach. It combines several HTML extraction methods and can use Apify for JavaScript-heavy sites.

Key features

  1. 01Extract names, titles, companies, and bios
  2. 02Try multiple HTML extraction strategies
  3. 03Use Apify for JavaScript-heavy sites
  4. 04Export JSON, CSV, or summary

Use cases

  1. 01Build a list of conference speakers
  2. 02Collect speaker details for outreach

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/gooseworks-ai/goose-skills --skill conference-speaker-scraper

After installing, type/conference-speaker-scraperin Claude Code to confirm it loaded.

Bundled filesscripts/scrape_speakers.pyskill.meta.json

Details

From the project README

Conference Speaker Scraper

Extract speaker names, titles, companies, and bios from conference website /speakers pages. Supports direct HTML scraping with multiple extraction strategies, plus Apify fallback for JS-heavy sites.

Quick Start

No API key needed for direct scraping mode.

# Scrape speakers from a conference page
python3 skills/conference-speaker-scraper/scripts/scrape_speakers.py \
  --url "https://example.com/speakers"

# Use Apify for JS-heavy sites
python3 skills/conference-speaker-scraper/scripts/scrape_speakers.py \
  --url "https://example.com/speakers" --mode apify

# Custom conference name (otherwise inferred from URL)
python3 skills/conference-speaker-scraper/scripts/scrape_speakers.py \
  --url "https://example.com/speakers" --conference "Sage Future 2026"

# Output formats
python3 skills/conference-speaker-scraper/scripts/scrape_speakers.py --url URL --output json     # default
python3 skills/conference-speaker-scraper/scripts/scrape_speakers.py --url URL --output csv
python3 skills/conference-speaker-scraper/scripts/scrape_speakers.py --url URL --output summary

How It Works

Direct Mode (default)

Fetches the page HTML and tries multiple extraction strategies in order, using whichever returns the most results:

  1. Strategy A -- CSS class hints: Looks for speaker cards with class names containing "speaker", "presenter", "faculty", "panelist", "team-member"
  2. Strategy B -- Heading + paragraph patterns: Looks for repeated <h2>/<h3> + <p> structures
  3. Strategy C -- JSON-LD structured data: Checks for <script type="application/ld+json"> with speaker data
  4. Strategy D -- Platform embeds: Detects Sched.com/Sessionize patterns used by many conferences
Apify Mode

Uses apify/cheerio-scraper actor with a custom page function that targets common speaker card selectors. Standard POST/poll/GET dataset pattern.

CLI Reference

FlagDefaultDescription
--urlrequiredConference speakers page URL
--conferenceinferredConference name (otherwise inferred from URL domain)
--modedirectdirect (HTML scraping) or apify (Apify cheerio scraper)
--outputjsonOutput format: json, csv, or summary
--tokenenv varApify token (only needed for apify mode)
--timeout300Max seconds for Apify run

Output Schema

{
  "name": "Jane Smith",
  "title": "VP of Finance",
  "company": "Acme Corp",
  "bio": "Jane leads the finance transformation at...",
  "linkedin_url": "https://linkedin.com/in/janesmith",
  "image_url": "https://...",
  "conference": "Sage Future 2026",
  "source_url": "https://sagefuture2026.com/speakers"
}

Cost

  • Direct mode: Free (no API, no tokens)
  • Apify mode: Uses apify/cheerio-scraper -- minimal Apify credits

Testing Notes

HTML scraping is inherently fragile across conference sites. The multi-strategy approach maximizes coverage, but JS-heavy sites will require Apify mode. When direct scraping returns 0 results, try --mode apify.

Comments

Markdown supported