Parsec · results/v4/v4_t_e1/cost_time/v4_t4_e1_results_table.md source
v4_t4_e1 -- T4 (merge) zero-shot sweep: cost & time
One row per task -- each task is exactly ONE baseline eval (--n-trials 5), zero-shot, no optimization step. Generated by scripts/v4_t4_e1/collect_cost_time.py; safe to re-run mid-sweep.
Progress: 34/34 tasks done. Cost so far: $42.78. Tokens so far: 13,601,491. Wall time so far: 295.2 min (4.92 hr). Average per task so far: $1.26, 8.7 min. Projected for the 0 remaining tasks at this average: $0.00, 0.00 hr more.
Summary table
| # | Task | Reward(val) | stderr | Trial rewards | Errored | Headroom | Cost($) | Tokens | Time(s) |
|---|---|---|---|---|---|---|---|---|---|
| 1 | cloud-024-guid-to-account | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 0.4164 | 133,729 | 273.4 |
| 2 | cloud-026-gpu-abuse-triage | 0.6133 | 0.1891 | 1.000, 1.000, 0.650, 0.000, 0.417 | 0 | ok | 0.4434 | 128,681 | 566.3 |
| 3 | cost-029-no-cost-rows-for-guid | 0.9600 | 0.0400 | 0.800, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 0.7137 | 226,728 | 238.2 |
| 4 | cost-030-threshold-not-an-anomaly | 0.7200 | 0.1171 | 0.580, 0.440, 1.000, 0.580, 1.000 | 0 | ok | 0.7795 | 234,863 | 360.4 |
| 5 | icinga-010-stuck-anarchysubjects | 0.4390 | 0.0343 | 0.495, 0.495, 0.355, 0.495, 0.355 | 0 | ok | 1.6261 | 520,302 | 413.7 |
| 6 | icinga-011-aap2-job-status-alert | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 1.0590 | 328,448 | 403.0 |
| 7 | icinga-013-acknowledged-not-an-issue | 0.6400 | 0.0000 | 0.640, 0.640, 0.640, 0.640, 0.640 | 0 | ok | 0.5368 | 171,800 | 177.5 |
| 8 | icinga-014-check-script-path-moved | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 1.7243 | 541,697 | 611.3 |
| 9 | platform-001-ee-entrypoint-rca | 0.7600 | 0.0443 | 0.760, 0.760, 0.900, 0.620, 0.760 | 0 | ok | 0.5822 | 172,657 | 1134.5 |
| 10 | platform-002-collection-not-found-rca | 0.9567 | 0.0433 | 1.000, 1.000, 1.000, 1.000, 0.783 | 0 | ok | 0.5702 | 171,849 | 1065.2 |
| 11 | platform-003-tojson-dict-literal-rca | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 0.5282 | 156,611 | 427.5 |
| 12 | platform-004-events-then-config | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 0.6483 | 201,187 | 495.3 |
| 13 | platform-005-wrong-owner-trap | 0.9120 | 0.0880 | 1.000, 1.000, 0.560, 1.000, 1.000 | 0 | saturated | 2.0414 | 662,262 | 388.2 |
| 14 | platform-007-directory-path-fetch | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 0.4527 | 142,162 | 348.1 |
| 15 | platform-008-log-does-not-say | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 0.5115 | 155,588 | 846.2 |
| 16 | platform-022-job-on-no-controller | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 0.4586 | 141,393 | 351.1 |
| 17 | platform-023-splunk-guid-no-events | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 0.4824 | 144,948 | 328.7 |
| 18 | platform-031-helm-url-not-a-timeout | 0.8655 | 0.0928 | 0.502, 1.000, 0.912, 0.912, 1.000 | 0 | ok | 7.2532 | 2,356,745 | 1028.5 |
| 19 | platform-032-shared-secret-not-a-registry-outage | 0.6450 | 0.0327 | 0.662, 0.662, 0.575, 0.575, 0.750 | 0 | ok | 4.1784 | 1,359,945 | 1295.8 |
| 20 | platform-033-schema-change-not-the-oom | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 2.3041 | 736,042 | 509.7 |
| 21 | platform-034-rate-limit-not-an-outage | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 0.7749 | 221,262 | 1686.1 |
| 22 | cloud-025-gcp-project-status | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 1.0586 | 340,658 | 306.8 |
| 23 | cloud-027-azure-pool-database-trap | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 0.7074 | 227,259 | 280.4 |
| 24 | cost-028-highest-spend-provider | 0.4000 | 0.0000 | 0.400, 0.400, 0.400, 0.400, 0.400 | 0 | ok | 0.8011 | 252,276 | 244.2 |
| 25 | icinga-012-nodes-ready-threshold | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 0.8901 | 270,555 | 319.4 |
| 26 | icinga-015-problems-by-severity | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 1.1088 | 350,938 | 288.3 |
| 27 | icinga-016-hosts-most-problems | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 0.8446 | 268,145 | 221.3 |
| 28 | icinga-017-invented-action-trap | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 1.2685 | 403,439 | 409.9 |
| 29 | platform-006-version-tag-not-a-ref | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 0.4775 | 145,933 | 348.3 |
| 30 | platform-009-catalog-item-no-config | 0.8250 | 0.0000 | 0.825, 0.825, 0.825, 0.825, 0.825 | 0 | ok | 0.5074 | 157,064 | 657.1 |
| 31 | platform-018-find-jobs-window | 0.0280 | 0.0280 | 0.000, 0.140, 0.000, 0.000, 0.000 | 0 | ok | 2.5566 | 835,103 | 398.0 |
| 32 | platform-019-anarchysubject-state | 0.3200 | 0.1700 | 0.150, 0.150, 0.150, 0.150, 1.000 | 0 | ok | 2.1212 | 690,591 | 425.8 |
| 33 | platform-020-ocpv-vm-inventory | 0.3400 | 0.1691 | 0.100, 1.000, 0.100, 0.300, 0.200 | 0 | ok | 1.3000 | 407,534 | 631.9 |
| 34 | platform-021-babylon-missing-cluster-trap | 1.0000 | 0.0000 | 1.000, 1.000, 1.000, 1.000, 1.000 | 0 | saturated | 1.0490 | 343,097 | 229.7 |