SkillsBench · results/transfer-eval-8fold/heatmap.md source

transfer-eval-8fold — heatmap

Zero-shot skill transfer, 8 folds, generated 2026-09-02.

Read this before the table
  • Every fold reports best_id='seed', test_reward == test_baseline_reward, test_delta == 0.0. This is an artifact of --max-iterations 0: the transfer project's seed IS the frozen donor skill and is the only artifact evaluated, so cap-evolve compares it against itself. Real transfer effect = transfer_reward - native_seed.
  • For tasks with status KILLED_ceiling, native_optimized is the in-loop val score (results.json 'best'), not a held-out test score, because final_test is null - the run was stopped on saturation before a held-out eval ran.
  • 8 folds over 5 tasks in 2 domains; directional only.
#train tasktest taskjobstatus transfer rewardnative seednative optimizedtransfer − native seed
1shock-analysis-demandshock-analysis-supply555427DONE0.100.000.20+0.10
2shock-analysis-demandweighted-gdp-calc540431DONE0.200.801.00 ‡-0.60
3shock-analysis-supplyshock-analysis-demand543167DONE0.000.000.900.00
4shock-analysis-supplyweighted-gdp-calc540433DONE0.300.801.00 ‡-0.50
5weighted-gdp-calcshock-analysis-demand543168DONE0.000.000.900.00
6weighted-gdp-calcshock-analysis-supply543169DONE0.100.000.20+0.10
7exam-block-sequencingparatransit-routing543170DONE0.200.001.00 ‡+0.20
8paratransit-routingexam-block-sequencing543171DONE0.400.101.00 ‡+0.30

‡ in-loop val score, not a held-out test score (task is KILLED_ceiling in results.json).

← back to the fold-by-fold writeup