SkillsBench · HOME.md source

SkillsBench experiments

Three experiments cap-evolve has run against BenchFlow's SkillsBench (87 tasks, 8 categories). Each card is level 1 — a row on the Benchmarks dashboard — with links onward to level 2 (a heatmap or per-task table) and, where it exists, level 3 (a per-task report).

Level 3 is not fully on-site yet

Per-task reports below are rendered from the skillberry-runs/skillsbench repo, but the skill packages they discuss (seed/, best/, PROCESS.md under artifacts/ — about 27 MB) are not rendered on this site. Each report page links to them in that repo on GitHub, marked GitHub.

task-by-task — one skill per task (87 tasks)

EvoSkills-style: each task gets its own optimized skill, seeded from the task's own environment/skills/. The full 3-level drill-down exists for this experiment.

transfer-eval-8fold — zero-shot skill transfer (8 folds)

Does a skill optimized for one task carry over to a different task, evaluated zero-shot (--max-iterations 0, no adaptation)? No level 3 — this experiment doesn't produce per-task skill artifacts of its own, only a comparison table.

runnable-subset-43 — whole-suite optimization (43 tasks)

One shared skill package optimized across all 43 tasks that actually run on our CCC podman setup. Cancelled at iteration 4 (optimizer hang); best accepted candidate is cand_0002.

More