SpreadsheetBench · reports/results_tables.md source
Version A. Sorted by mean test gain (champion − same-window seed)
| # | System | Train (80) | Val (40) | Test (280): seed → champion (gain) | Gain SD across seeds | Test gain per seed (s0 / s1 / s2) |
|---|---|---|---|---|---|---|
| 1 | GEPA-full | n/a | 0.706 → 0.872 (+0.167) | 0.638 → 0.767 (+0.129) | 0.034 | +0.145 / +0.090 / +0.152 |
| 2 | SkillOpt | n/a | 0.697 → 0.886 (+0.189) | 0.640 → 0.759 (+0.119) | 0.018 | +0.098 / +0.123 / +0.134 |
| 3 | GEPA | n/a | 0.716 → 0.867 (+0.150) | 0.614 → 0.732 (+0.118) | 0.054 | +0.179 / +0.100 / +0.075 |
| 4 | WikiSkill | 0.667 → 0.690 (+0.023), 2 of 3 seeds | 0.692 → 0.831 (+0.139) | 0.623 → 0.719 (+0.097) | 0.016 | +0.089 / +0.086 / +0.115 |
| 5 | MIPROv2 | n/a | 0.719 → 0.817 (+0.097) | 0.637 → 0.734 (+0.096) | 0.036 | +0.093 / +0.135 / +0.062 |
| 6 | CapEvolve v2 | 0.727 → 0.813 (+0.086) | 0.730 → 0.860 (+0.130) | 0.631 → 0.726 (+0.095) | 0.041 | +0.075 / +0.068 / +0.143 |
| 7 | CapEvolve v1 | 0.670 → 0.763 (+0.092) | 0.686 → 0.814 (+0.128) | 0.641 → 0.730 (+0.090) | 0.045 | +0.142 / +0.066 / +0.061 |
| 8 | EvoSkill | 0.688 → 0.688 (+0.000), 1 of 3 seeds | 0.712 → 0.826 (+0.114) | 0.643 → 0.732 (+0.087) | 0.081 | -0.006* / +0.143 / +0.123 |
| 9 | Trace2Skill | 0.600 → 0.600 (+0.000), 1 of 3 seeds | 0.700 → 0.719 (+0.019) | 0.642 → 0.699 (+0.054) | 0.048 | +0.000* / +0.069 / +0.093 |
| 10 | Meta-Harness | 0.683 → 0.717 (+0.033) | 0.731 → 0.806 (+0.075) | 0.640 → 0.692 (+0.052) | 0.010 | +0.041 / +0.060 / +0.055 |
| 11 | ACE | n/a | n/a | 0.616 → 0.656 (+0.040) | 0.057 | +0.015 / +0.105 / +0.000* |
Version B. Sorted by mean absolute test score of the champion
| # | System | Train (80) | Val (40) | Test (280): seed → champion (gain) | Gain SD across seeds | Test gain per seed (s0 / s1 / s2) |
|---|---|---|---|---|---|---|
| 1 | GEPA-full | n/a | 0.706 → 0.872 (+0.167) | 0.638 → 0.767 (+0.129) | 0.034 | +0.145 / +0.090 / +0.152 |
| 2 | SkillOpt | n/a | 0.697 → 0.886 (+0.189) | 0.640 → 0.759 (+0.119) | 0.018 | +0.098 / +0.123 / +0.134 |
| 3 | MIPROv2 | n/a | 0.719 → 0.817 (+0.097) | 0.637 → 0.734 (+0.096) | 0.036 | +0.093 / +0.135 / +0.062 |
| 4 | GEPA | n/a | 0.716 → 0.867 (+0.150) | 0.614 → 0.732 (+0.118) | 0.054 | +0.179 / +0.100 / +0.075 |
| 5 | EvoSkill | 0.688 → 0.688 (+0.000), 1 of 3 seeds | 0.712 → 0.826 (+0.114) | 0.643 → 0.732 (+0.087) | 0.081 | -0.006* / +0.143 / +0.123 |
| 6 | CapEvolve v1 | 0.670 → 0.763 (+0.092) | 0.686 → 0.814 (+0.128) | 0.641 → 0.730 (+0.090) | 0.045 | +0.142 / +0.066 / +0.061 |
| 7 | CapEvolve v2 | 0.727 → 0.813 (+0.086) | 0.730 → 0.860 (+0.130) | 0.631 → 0.726 (+0.095) | 0.041 | +0.075 / +0.068 / +0.143 |
| 8 | WikiSkill | 0.667 → 0.690 (+0.023), 2 of 3 seeds | 0.692 → 0.831 (+0.139) | 0.623 → 0.719 (+0.097) | 0.016 | +0.089 / +0.086 / +0.115 |
| 9 | Trace2Skill | 0.600 → 0.600 (+0.000), 1 of 3 seeds | 0.700 → 0.719 (+0.019) | 0.642 → 0.699 (+0.054) | 0.048 | +0.000* / +0.069 / +0.093 |
| 10 | Meta-Harness | 0.683 → 0.717 (+0.033) | 0.731 → 0.806 (+0.075) | 0.640 → 0.692 (+0.052) | 0.010 | +0.041 / +0.060 / +0.055 |
| 11 | ACE | n/a | n/a | 0.616 → 0.656 (+0.040) | 0.057 | +0.015 / +0.105 / +0.000* |