Table 2 · Parsec v6 · cost, tokens and compute time

The task-wise path built its bundle for 42–44% of Global's cost

2a is what each arm spent to produce its deployable bundle, with no test split. 2b is what it costs to measure a frozen bundle, at 5 trials per task for every arm. The shared Seed measurements are in neither, since every arm reuses them.

2a · Cost to build the deployable bundle

Cost (USD) Tokens (millions) Compute time (hours)
Arm EvalOptim.Total EvalOptim.Total EvalOptim.Total Eval trials
Task-wise (T)
T-opt34 per-task searches $38$281$319 11.3283.4294.7 13.819.433.1 310
T-mrg · merge by orderT-opt + 10 s script $38$281+$0$319 11.3283.4+0294.7 13.819.4+10 s33.1 310
T-mrg · smart mergeT-opt + LLM merger $38$295+$14$334 11.3293.6+10.2304.9 13.820.2+0.834.0 310
T-mrg · non-conflicting hunksT-opt + 10 s script $38$281+$0$319 11.3283.4+0294.7 13.819.4+10 s33.1 310
Global (G)
Global · cand_0005train screens + validation gates, 6 iterations $403$355$758 124.9155.0279.8 164.28.9173.1 2,223

Global's evaluation: train screens and incumbent sweeps $109 (585 trials), validation gates $294 (1,638 trials).

2b · Cost to measure a frozen bundle, 5 trials per task

Test, 66 tasks Train, 67 tasks
Arm $TokensHours $TokensHours
T-mrg · merge by order$7824.4M8.7$7022.1M7.8
T-mrg · smart merge$10834.2M9.3$9329.5M8.1
T-mrg · non-conflicting hunks$6620.4M9.6$5517.1M8.1
Global · cand_0005first 5 of its 10 test trials$7021.8M24.7— train reward comes from its search sweeps

At equal trials, measuring costs about the same in both arms; larger bundles cost more per trial (Table 3). Global's full 10-trial test cost $138.

Reading the tables

  • Build cost: $319–334 for T against $758 for Global, so T is 42–44% of Global. T spends 88% on the optimizer; Global spends 53% on evaluation, mostly validation gates.
  • Tokens are not an effort measure. Totals look alike, but optimizer tokens are about 95% (T) and 97% (Global) cache reads: the optimizer re-reading its context every turn. Cache reads are billed at a tenth of input price yet counted at full weight. Evaluation tokens cost about $3.2 per million; optimizer tokens $0.99 (T) and $2.29 (Global). Compare effort in dollars.
  • Compute time: Global's build is 5.2× T's. Global ran 7.2× the evaluation trials, and its trials ran longer on average (266 s vs 160 s), partly because of slow simulator calls during its run: 9% of trials logged in the Global run took over 10 minutes. Its 5-trial test took 24.7 h of compute against 8.7–9.6 h per merge, for similar dollars.

Definitions

  • Eval is the target agent running tasks; Optim. is the optimizer, plus the smart merge's LLM merger. The other two merges are 10-second scripts with no LLM.
  • Dollars and tokens are the per-trial records' own values, plus recorded optimizer spend. The smart merge's $14 is a list-price estimate from its measured tokens.
  • Compute time sums each trial's agent run and each optimizer call: effort, not wall clock. Global's run took 66 h of wall clock.
  • Not counted: shared Seed measurements ($215); Global's post-run validation remeasurement of cand_0002 ($53); a harness step that re-scored some T-mrg tasks a second time ($14–29 per method, unused).