2026-07-20 04:00 UTCOriginal source2 min readUpdated: 2026-07-20 08:26 UTC

SkillCorpus: Consolidating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents

SkillCorpus aggregates, curates, and evaluates over 96,000 open-source LLM agent skills from ~821,000 candidates, using a 16-class taxonomy and quality facets. Integrated with a retrieval-and-selection stack, it achieves consistent gains across benchmarks, with the largest improvement of +7.5 percentage points on SkillsBench.

SourcearXiv Computational LinguisticsAuthor: Yanze Wang, Pengfei Yao, Tianyi Sun, Chuanrui Hu, Yan Xiao, Yunyun Han, Jun Sun, Yafeng Deng

-->

[Submitted on 17 Jul 2026]

Title:SkillCorpus: Consolidating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents

View a PDF of the paper titled SkillCorpus: Consolidating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents, by Yanze Wang and 7 other authors

View PDF HTML (experimental)

Abstract:Agent skills, this http URL files that package reusable procedural knowledge for an LLM agent, are a popular mechanism for extending agent capabilities. Public repositories now host them in large and growing numbers, yet these artifacts are fragmented, redundant, and uneven in quality, and their value in practice is unclear. A core question remains open, namely how to consolidate this open-source this http URL ecosystem into a single usable corpus, and what bounds its benefit on real-world agent tasks. We present SkillCorpus, a framework that aggregates, curates, matches, and evaluates the open skill ecosystem at scale. It filters ~821,000 crawled skills through a multi-stage pipeline into 96,401 skills organised by a 16-class taxonomy and three quality facets (utility, robustness, safety), and pairs them with a fine-tuned retrieval-and-selection stack that matches task-relevant skills. We evaluate end-to-end across three benchmarks (SkillsBench, GDPVal, QwenClawBench), two harnesses, and two open backbones with a frontier robustness check. Integrating SkillCorpus yields consistent gains across all three benchmarks, largest on SkillsBench (+7.5 pp). An operational analysis traces the gains to a coverage boundary and a harness boundary. SkillCorpus is, to our knowledge, the first end-to-end account of when a curated, retrieval-served community corpus improves real agent tasks, and where it does not. The dataset, models, and code will be released upon acceptance.

Subjects:

Computation and Language (cs.CL)

Cite as: arXiv:2607.15557 [cs.CL]

(or arXiv:2607.15557v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2607.15557

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Yanze Wang [view email] [v1] Fri, 17 Jul 2026 01:55:17 UTC (1,458 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled SkillCorpus: Consolidating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents, by Yanze Wang and 7 other authors

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.CL

new | recent | 2026-07

Change to browse by:

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)