Testing platform for ZShot
| Template | Total Runs | Pass | Fail | Pass Rate | Avg Tokens In | Avg Tokens Out | Tokens In per Pass |
|---|---|---|---|---|---|---|---|
| benefits_hardship_claim | 32 | 16 | 16 | 50.0% | 898,848 | 19,789 | 1,235,917 |
| doom_destroy_a_barrel | 15 | 7 | 8 | 46.7% | 208,211 | 5,319 | 446,166 |
| parcel_redelivery | 281 | 116 | 165 | 41.3% | 181,211 | 15,199 | 412,846 |
| recipe_site_favorite_a_recipe | 16 | 8 | 8 | 50.0% | 15,315 | 609 | 15,315 |
Token figures count runs recorded with --llm-usage. Tokens in per pass is every metered run's input over the metered runs that passed.
| ID | Template | Label | Result | Furthest | Tokens | Est. Cost | Took | Created |
|---|---|---|---|---|---|---|---|---|
| 250 | doom_destroy_a_barrel | haiku@doomrec | pass | — | 208,747 in / 16,340 out | $0.029 | 2m 16s | 2026-10-12 12:21 |
| 249 | doom_destroy_a_barrel | haiku@doomrec | pass | — | 119,787 in / 8,628 out | $0.016 | 1m 22s | 2026-10-12 12:21 |
| 248 | doom_destroy_a_barrel | haiku@doomrec | pass | — | 104,561 in / 6,947 out | $0.014 | 1m 7s | 2026-10-12 12:21 |
| 247 | doom_destroy_a_barrel | clef@timed-1012 | fail | — | 361,668 in / 0 out | $0.087 | 2m 20s | 2026-10-12 12:11 |
| 246 | doom_destroy_a_barrel | decisions-vision@timed-1012 | fail | — | 32,154 in / 0 out | $0.003 | 11s | 2026-10-12 12:11 |
| 245 | doom_destroy_a_barrel | decisions-vision@timed-1012 | fail | — | 334,410 in / 0 out | $0.033 | 1m 5s | 2026-10-12 12:11 |
| 244 | doom_destroy_a_barrel | clef@timed-1012 | fail | — | 361,612 in / 0 out | $0.087 | 2m 27s | 2026-10-12 12:11 |
| 243 | parcel_redelivery | scout-clef@regress-1011 | pass | done | 335,354 in / 3,963 out | $0.350 | 2m 7s | 2026-10-11 18:29 |
| 242 | parcel_redelivery | scout-decisions@regress-1011 | fail | confirm | 709,826 in / 2,027 out | $0.199 | 1m 52s | 2026-10-11 18:29 |
| 241 | parcel_redelivery | scout-sonnet-opus@regress-1011 | fail | human | 189,056 in / 9,129 out | $0.200 | 56s | 2026-10-11 18:29 |
| 240 | parcel_redelivery | scout-sonnet@regress-1011 | fail | preferences | 737,057 in / 18,891 out | $0.508 | 2m 3s | 2026-10-11 18:25 |
| 239 | parcel_redelivery | gemini@regress-1011 | fail | human | 30,782 in / 8,866 out | $0.056 | 2m 18s | 2026-10-11 18:25 |
| 238 | parcel_redelivery | deepseek@regress-1011 | fail | preferences | 123,990 in / 33,166 out | $0.070 | 4m 8s | 2026-10-11 18:25 |
| 237 | parcel_redelivery | luna@regress-1011 | fail | done | 104,710 in / 3,589 out | $0.012 | 1m 25s | 2026-10-11 18:25 |
| 236 | parcel_redelivery | gpt@regress-1011 | pass | done | 520,743 in / 4,664 out | $5.41 | 4m 47s | 2026-10-11 18:20 |
| 235 | parcel_redelivery | sonnet@regress-1011 | pass | done | 179,916 in / 5,634 out | $0.386 | 1m 37s | 2026-10-11 18:20 |
| 234 | parcel_redelivery | haiku@regress-1011 | fail | human | 289,764 in / 28,773 out | $0.041 | 3m 34s | 2026-10-11 18:20 |
| 233 | parcel_redelivery | opus@regress-1011 | pass | done | 174,059 in / 5,059 out | $0.736 | 2m 38s | 2026-10-11 18:20 |
| 232 | parcel_redelivery | scout-decisions@rel5-1011 | pass | done | 192,949 in / 2,471 out | $0.227 | 1m 5s | 2026-10-11 12:27 |
| 231 | parcel_redelivery | scout-decisions@rel5-1011 | pass | done | 300,562 in / 1,887 out | $0.173 | 1m 7s | 2026-10-11 12:26 |
| 230 | parcel_redelivery | scout-decisions@rel5-1011 | fail | done | 94,405 in / 800 out | $0.062 | 37s | 2026-10-11 12:25 |
| 229 | parcel_redelivery | luna-openrouter@rel5-1010 | pass | done | 256,487 in / 11,675 out | $0.031 | 4m 0s | 2026-10-11 02:07 |
| 228 | parcel_redelivery | scout-decision1@rel5-1010 | fail | done | 86,997 in / 1,583 out | $0.095 | 52s | 2026-10-11 02:06 |
| 227 | parcel_redelivery | luna-openrouter@rel5-1010 | fail | done | 80,342 in / 2,914 out | $0.009 | 1m 28s | 2026-10-11 02:05 |
| 226 | parcel_redelivery | scout-decision1@rel5-1010 | fail | confirm | 280,791 in / 7,796 out | $0.636 | 2m 24s | 2026-10-11 02:03 |
| 225 | parcel_redelivery | scout-jev-1.13-openrouter@rel5-1010 | pass | done | 246,675 in / 10,020 out | $0.286 | 1m 30s | 2026-10-11 01:44 |
| 224 | parcel_redelivery | scout-microsoft-decision-1-openrouter@rel5-1010 | fail | done | 81,497 in / 1,294 out | $0.086 | 43s | 2026-10-11 01:44 |
| 223 | parcel_redelivery | scout-jev-1.13-openrouter@rel5-1010 | pass | done | 239,810 in / 9,647 out | $0.269 | 1m 24s | 2026-10-11 01:42 |
| 222 | parcel_redelivery | scout-microsoft-decision-1-openrouter@rel5-1010 | pass | done | 151,508 in / 3,745 out | $0.271 | 1m 20s | 2026-10-11 01:41 |
| 221 | parcel_redelivery | luna@rel5-1010 | pass | done | 194,819 in / 5,217 out | $0.021 | 2m 25s | 2026-10-10 20:10 |
| 220 | parcel_redelivery | scout-clef@rel5-1010 | pass | done | 397,084 in / 14,192 out | $0.657 | 3m 27s | 2026-10-10 20:06 |
| 219 | parcel_redelivery | scout-sonnet-opus@rel5-1010 | pass | done | 221,811 in / 8,855 out | $0.274 | 1m 13s | 2026-10-10 20:05 |
| 218 | parcel_redelivery | haiku@rel5-1010 | pass | done | 435,817 in / 28,254 out | $0.058 | 3m 31s | 2026-10-10 20:02 |
| 217 | parcel_redelivery | deepseek@rel5-1010 | fail | preferences | 131,763 in / 38,726 out | $0.078 | 4m 10s | 2026-10-10 19:57 |
| 216 | parcel_redelivery | gpt@rel5-1010 | pass | done | 298,469 in / 2,710 out | $3.08 | 3m 10s | 2026-10-10 19:54 |
| 215 | parcel_redelivery | opus@rel5-1010 | pass | done | 101,085 in / 2,978 out | $0.464 | 1m 17s | 2026-10-10 19:53 |
| 214 | parcel_redelivery | sonnet@rel5-1010 | pass | done | 147,774 in / 4,359 out | $0.339 | 1m 18s | 2026-10-10 19:52 |
| 213 | parcel_redelivery | luna@rel5-1010 | fail | done | 138,757 in / 2,617 out | $0.015 | 1m 35s | 2026-10-10 19:50 |
| 212 | parcel_redelivery | scout-clef@rel5-1010 | fail | slot | 680,932 in / 13,012 out | $1.10 | 4m 52s | 2026-10-10 19:45 |
| 211 | parcel_redelivery | scout-sonnet-opus@rel5-1010 | pass | done | 176,649 in / 6,708 out | $0.218 | 58s | 2026-10-10 19:44 |
| 210 | parcel_redelivery | haiku@rel5-1010 | pass | done | 107,090 in / 6,913 out | $0.014 | 1m 8s | 2026-10-10 19:43 |
| 209 | parcel_redelivery | deepseek@rel5-1010 | fail | preferences | 72,692 in / 11,205 out | $0.031 | 1m 51s | 2026-10-10 19:41 |
| 208 | parcel_redelivery | gpt@rel5-1010 | pass | done | 263,726 in / 2,028 out | $2.71 | 2m 30s | 2026-10-10 19:39 |
| 207 | parcel_redelivery | opus@rel5-1010 | pass | done | 131,895 in / 3,310 out | $0.594 | 1m 27s | 2026-10-10 19:37 |
| 206 | parcel_redelivery | sonnet@rel5-1010 | pass | done | 157,037 in / 4,630 out | $0.360 | 1m 21s | 2026-10-10 19:36 |
| 205 | parcel_redelivery | luna@rel-1010 | fail | done | 222,609 in / 7,136 out | $0.026 | 3m 11s | 2026-10-10 19:19 |
| 204 | parcel_redelivery | scout-clef@rel-1010 | fail | human | 108,753 in / 2,273 out | $0.149 | 1m 5s | 2026-10-10 19:18 |
| 203 | parcel_redelivery | scout-sonnet-opus@rel-1010 | fail | human | 64,405 in / 4,139 out | $0.092 | 34s | 2026-10-10 19:18 |
| 202 | parcel_redelivery | haiku@rel-1010 | fail | human | 31,279 in / 2,111 out | $0.004 | 24s | 2026-10-10 19:17 |
| 201 | parcel_redelivery | deepseek@rel-1010 | fail | human | 42,722 in / 9,394 out | $0.020 | 1m 40s | 2026-10-10 19:15 |