mcpField
S
scientific-llm-benchmarks

scientific-llm-benchmarks

github.com/akillnessTesting & quality5396 installsUpdated 4d ago
Repository

About scientific-llm-benchmarks

A reference skill for researchers and developers comparing LLM performance on scientific reasoning and discovery. It organizes benchmark references and YAML data across STEM fields, with scripts for rebuilding lists and fetching dataset examples.

Key features

  1. 01Benchmarks across scientific domains
  2. 02Markdown references and YAML data
  3. 03Regenerate benchmark lists
  4. 04Fetch dataset sample rows

Use cases

  1. 01Find evaluations for a scientific LLM
  2. 02Compare benchmarks across STEM fields
  3. 03Fetch sample rows from benchmark datasets

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/akillness/jeo-skills --skill scientific-llm-benchmarks

After installing, type/scientific-llm-benchmarksin Claude Code to confirm it loaded.

Bundled files

References and scripts installed along with SKILL.md.

data/
benchmarks.yaml
examples.json
references/
benchmarks.md
scripts/
fetch_examples.py
generate_readme.py

Details

From the project README

Scientific LLM Benchmarks

This skill provides references to benchmarks used for evaluating large language models on scientific reasoning and discovery. The data comes from the Awesome-Scientific-LLM-Benchmarks repository.

Contents

The complete benchmark list is stored locally within this skill:

  • References List: references/benchmarks.md
  • Data (YAML format): data/benchmarks.yaml

Benchmark Domains Covered

  • General / Multi-domain Science: Cross-disciplinary STEM reasoning benchmarks.
  • Mathematics: Arithmetic, competition, olympiad, and frontier / formal-proof mathematics.
  • Physics and Astronomy: Physics olympiad, graduate physics, computational physics, and astronomy.
  • Chemistry: Molecular property, reaction, retrosynthesis, safety, and chemical knowledge.
  • Materials Science: Crystals, materials property prediction, and materials-science knowledge.
  • Biology and Life Sciences: Genomics, proteins, bioinformatics agents, protocols, and research biology.
  • Agentic Science and AI Research: LLM agents that write research code, run data analyses, attempt autonomous discovery, and conduct ML/AI research.

Helper Scripts

Also included are python scripts inside scripts/:

  • generate_readme.py: Regenerates the markdown tables and list from data/benchmarks.yaml.
  • fetch_examples.py: Fetches real sample rows from HuggingFace dataset URLs specified in the dataset metadata.

Comments

Markdown supported