mcpField
A
alicloud-platform-multicloud-docs-api-benchmark

alicloud-platform-multicloud-docs-api-benchmark

github.com/cinienceTesting & quality396260 installsUpdated 11w ago
Repository

About alicloud-platform-multicloud-docs-api-benchmark

A skill for comparing documentation and API coverage for similar products across seven cloud providers. It runs a benchmark with configurable presets and scoring, prioritizing official sources and producing evidence and a report.

Key features

  1. 01Seven-cloud product comparisons
  2. 02Official-source discovery tiers
  3. 03Preset-based benchmark configuration
  4. 04Switchable scoring profiles
  5. 05Evidence and report artifacts

Use cases

  1. 01Compare serverless product documentation
  2. 02Benchmark LLM platform APIs
  3. 03Pin official links for stricter comparisons

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/cinience/alicloud-skills --skill alicloud-platform-multicloud-docs-api-benchmark

After installing, type/alicloud-platform-multicloud-docs-api-benchmarkin Claude Code to confirm it loaded.

Bundled files

References and scripts installed along with SKILL.md.

agents/
openai.yaml
references/
presets.json
review-rubric.md
scoring.json
scripts/
benchmark_multicloud_docs_api.py

Details

From the project README

Multi-Cloud Product Docs/API Benchmark

Use this skill when the user wants cross-cloud documentation/API comparison for similar products.

Supported clouds

  • Alibaba Cloud
  • AWS
  • Azure
  • GCP
  • Tencent Cloud
  • Volcano Engine
  • Huawei Cloud

Data source policy

  • L0 (highest): user-pinned official links via --<provider>-links
  • L1: machine-readable official metadata/source
    • GCP: Discovery API
    • AWS: API Models repository
    • Azure: REST API Specs repository
  • L2: official-domain constrained web discovery fallback
  • L3: insufficient discovery (low confidence)

Workflow

Run the benchmark script:

python skills/platform/docs/alicloud-platform-multicloud-docs-api-benchmark/scripts/benchmark_multicloud_docs_api.py --product "<product keyword>"

Example:

python skills/platform/docs/alicloud-platform-multicloud-docs-api-benchmark/scripts/benchmark_multicloud_docs_api.py --product "serverless"

LLM platform benchmark example (Bailian/Bedrock/Azure OpenAI/Vertex AI/Hunyuan/Ark/Pangu):

python skills/platform/docs/alicloud-platform-multicloud-docs-api-benchmark/scripts/benchmark_multicloud_docs_api.py --product "Bailian" --preset "llm-platform"

If --preset is omitted, script attempts to auto-match preset based on keyword.

Scoring weights can be switched by profile (see references/scoring.json):

python skills/platform/docs/alicloud-platform-multicloud-docs-api-benchmark/scripts/benchmark_multicloud_docs_api.py --product "Bailian" --preset "llm-platform" --scoring-profile "llm-platform"

Optional: pin authoritative links

Auto-discovery may miss pages. For stricter comparison, pass official links manually:

python skills/platform/docs/alicloud-platform-multicloud-docs-api-benchmark/scripts/benchmark_multicloud_docs_api.py \
  --product "object storage" \
  --aws-links "https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html" \
  --azure-links "https://learn.microsoft.com/azure/storage/blobs/"

Available manual flags:

  • --alicloud-links
  • --aws-links
  • --azure-links
  • --gcp-links
  • --tencent-links
  • --volcengine-links
  • --huawei-links

Each flag accepts comma-separated URLs.

Output policy

All artifacts must be written under:

output/alicloud-platform-multicloud-docs-api-benchmark/

Per run:

  • benchmark_evidence.json
  • benchmark_report.md

Reporting guidance

When answering the user:

  1. Show score ranking across all providers.
  2. Highlight top gaps (P0/P1/P2) and concrete fix actions.
  3. If discovery confidence is low, ask user to provide pinned links and rerun.

Validation

mkdir -p output/alicloud-platform-multicloud-docs-api-benchmark
for f in skills/platform/docs/alicloud-platform-multicloud-docs-api-benchmark/scripts/*.py; do
  python3 -m py_compile "$f"
done
echo "py_compile_ok" > output/alicloud-platform-multicloud-docs-api-benchmark/validate.txt

Pass criteria: command exits 0 and output/alicloud-platform-multicloud-docs-api-benchmark/validate.txt is generated.

Output And Evidence

  • Save artifacts, command outputs, and API response summaries under output/alicloud-platform-multicloud-docs-api-benchmark/.
  • Include key parameters (region/resource id/time range) in evidence files for reproducibility.

Prerequisites

  • Configure least-privilege Alibaba Cloud credentials before execution.
  • Prefer environment variables: ALICLOUD_ACCESS_KEY_ID, ALICLOUD_ACCESS_KEY_SECRET, optional ALICLOUD_REGION_ID.
  • If region is unclear, ask the user before running mutating operations.

References

  • Rubric: references/review-rubric.md

Comments

Markdown supported