Testing platform for ZShot
| Template | Total Runs | Pass | Fail | Pass Rate | Avg Tokens In | Avg Tokens Out | Tokens In per Pass |
|---|---|---|---|---|---|---|---|
| benefits_hardship_claim | 32 | 16 | 16 | 50.0% | 898,848 | 19,789 | 1,235,917 |
| doom_destroy_a_barrel | 15 | 7 | 8 | 46.7% | 208,211 | 5,319 | 446,166 |
| parcel_redelivery | 281 | 116 | 165 | 41.3% | 181,211 | 15,199 | 412,846 |
| recipe_site_favorite_a_recipe | 16 | 8 | 8 | 50.0% | 15,315 | 609 | 15,315 |
Token figures count runs recorded with --llm-usage. Tokens in per pass is every metered run's input over the metered runs that passed.
| ID | Template | Label | Result | Furthest | Tokens | Est. Cost | Took | Created |
|---|---|---|---|---|---|---|---|---|
| 200 | parcel_redelivery | gpt@rel-1010 | pass | done | 301,630 in / 2,244 out | $3.12 | 3m 0s | 2026-10-10 19:12 |
| 199 | parcel_redelivery | opus@rel-1010 | pass | done | 154,291 in / 4,087 out | $0.699 | 1m 56s | 2026-10-10 19:11 |
| 198 | parcel_redelivery | sonnet@rel-1010 | fail | human | 29,421 in / 1,524 out | $0.074 | 31s | 2026-10-10 19:10 |
| 197 | parcel_redelivery | scout-clef@rel-1010 | pass | done | 252,309 in / 2,138 out | $0.414 | 1m 47s | 2026-10-10 19:08 |
| 196 | parcel_redelivery | scout-sonnet-opus@rel-1010 | fail | human | 64,740 in / 4,642 out | $0.105 | 41s | 2026-10-10 19:08 |
| 195 | parcel_redelivery | haiku@rel-1010 | fail | human | 31,311 in / 2,078 out | $0.004 | 24s | 2026-10-10 19:07 |
| 194 | parcel_redelivery | deepseek@rel-1010 | fail | preferences | 181,454 in / 69,079 out | $0.128 | 7m 15s | 2026-10-10 19:00 |
| 193 | parcel_redelivery | gpt@rel-1010 | pass | done | 211,153 in / 1,532 out | $2.19 | 2m 5s | 2026-10-10 18:58 |
| 192 | parcel_redelivery | opus@rel-1010 | pass | done | 148,255 in / 6,037 out | $0.714 | 2m 12s | 2026-10-10 18:56 |
| 191 | parcel_redelivery | sonnet@rel-1010 | fail | human | 37,094 in / 1,535 out | $0.090 | 30s | 2026-10-10 18:55 |
| 190 | parcel_redelivery | luna@rel-1010 | pass | done | 102,772 in / 3,065 out | $0.012 | 1m 44s | 2026-10-10 18:53 |
| 189 | parcel_redelivery | haiku@visual-memory-1010 | pass | done | 145,002 in / 8,521 out | $0.019 | 1m 38s | 2026-10-10 10:29 |
| 188 | parcel_redelivery | scout-clef@visual-memory-1010 | pass | done | 152,412 in / 2,384 out | $0.209 | 1m 27s | 2026-10-10 10:28 |
| 187 | parcel_redelivery | scout-sonnet-opus@visual-memory-1010 | pass | done | 177,068 in / 6,712 out | $0.262 | 1m 26s | 2026-10-10 10:27 |
| 186 | parcel_redelivery | deepseek@visual-memory-1010 | pass | done | 92,736 in / 11,788 out | $0.040 | 1m 58s | 2026-10-10 10:25 |
| 185 | parcel_redelivery | gpt@visual-memory-1010 | pass | done | 257,742 in / 2,083 out | $2.68 | 3m 18s | 2026-10-10 10:21 |
| 184 | parcel_redelivery | opus@visual-memory-1010 | pass | done | 136,653 in / 3,552 out | $0.618 | 1m 53s | 2026-10-10 10:19 |
| 183 | parcel_redelivery | sonnet@visual-memory-1010 | pass | done | 152,043 in / 5,503 out | $0.359 | 1m 49s | 2026-10-10 10:18 |
| 182 | parcel_redelivery | haiku@visual-memory-1010 | fail | human | 28,426 in / 2,177 out | $0.004 | 25s | 2026-10-10 10:17 |
| 181 | parcel_redelivery | scout-clef@visual-memory-1010 | fail | done | 192,841 in / 1,105 out | $0.150 | 1m 39s | 2026-10-10 10:15 |
| 180 | parcel_redelivery | scout-sonnet-opus@visual-memory-1010 | fail | human | 70,439 in / 5,139 out | $0.112 | 48s | 2026-10-10 10:14 |
| 179 | parcel_redelivery | deepseek@visual-memory-1010 | fail | human | 64,949 in / 15,582 out | $0.036 | 2m 9s | 2026-10-10 10:12 |
| 178 | parcel_redelivery | gpt@visual-memory-1010 | pass | done | 263,330 in / 1,994 out | $2.71 | 3m 18s | 2026-10-10 10:09 |
| 177 | parcel_redelivery | opus@visual-memory-1010 | pass | done | 310,967 in / 15,389 out | $1.55 | 5m 4s | 2026-10-10 10:04 |
| 176 | parcel_redelivery | sonnet@visual-memory-1010 | fail | human | 31,297 in / 1,280 out | $0.075 | 31s | 2026-10-10 10:03 |
| 175 | parcel_redelivery | haiku@local-1009 | pass | done | 225,997 in / 15,669 out | $0.030 | 2m 21s | 2026-10-10 00:36 |
| 174 | parcel_redelivery | scout-clef@local-1009 | pass | done | 166,772 in / 1,726 out | $0.398 | 1m 30s | 2026-10-10 00:34 |
| 173 | parcel_redelivery | scout-sonnet-opus@local-1009 | pass | done | 208,007 in / 7,490 out | $0.290 | 1m 22s | 2026-10-10 00:33 |
| 172 | parcel_redelivery | deepseek@local-1009 | fail | preferences | 79,081 in / 13,676 out | $0.037 | 2m 28s | 2026-10-10 00:30 |
| 171 | parcel_redelivery | gpt@local-1009 | pass | done | 286,523 in / 2,516 out | $2.99 | 3m 44s | 2026-10-10 00:27 |
| 170 | parcel_redelivery | opus@local-1009 | pass | done | 130,264 in / 3,613 out | $0.593 | 1m 33s | 2026-10-10 00:25 |
| 169 | parcel_redelivery | sonnet@local-1009 | pass | done | 140,153 in / 4,934 out | $0.330 | 1m 29s | 2026-10-10 00:24 |
| 168 | doom_destroy_a_barrel | clef@doom-1009 | fail | — | 40,988 in / 0 out | $0.010 | 38s | 2026-10-09 19:57 |
| 167 | doom_destroy_a_barrel | clef@doom-1009 | fail | — | 40,988 in / 0 out | $0.010 | 36s | 2026-10-09 19:56 |
| 166 | doom_destroy_a_barrel | clef@doom-1009 | fail | — | 40,988 in / 0 out | $0.010 | 39s | 2026-10-09 19:55 |
| 165 | doom_destroy_a_barrel | scout-clef@doom-1009c | pass | — | 291,007 in / 10,864 out | $0.699 | 4m 10s | 2026-10-09 19:22 |
| 164 | doom_destroy_a_barrel | scout-clef@doom-1009c | pass | — | 436,440 in / 12,703 out | $1.31 | 5m 35s | 2026-10-09 19:17 |
| 163 | doom_destroy_a_barrel | scout-clef@doom-1009c | pass | — | 420,114 in / 11,173 out | $1.24 | 5m 22s | 2026-10-09 19:11 |
| 162 | doom_destroy_a_barrel | scout-clef@doom-1009b | active | — | — | — | — | 2026-10-09 19:10 |
| 161 | doom_destroy_a_barrel | scout-clef@doom-1009b | pass | — | 328,589 in / 13,135 out | $0.912 | 4m 58s | 2026-10-09 19:05 |
| 160 | doom_destroy_a_barrel | scout-clef@doom-1009 | fail | — | 1,112 in / 0 out | $0.000 | 3s | 2026-10-09 18:55 |
| 159 | parcel_redelivery | haiku-replay@1008 | pass | done | — | — | 12s | 2026-10-08 18:51 |
| 158 | parcel_redelivery | haiku@replay-1008 | pass | done | 131,639 in / 9,446 out | $0.018 | 1m 31s | 2026-10-08 18:49 |
| 157 | parcel_redelivery | haiku-replay@1008 | fail | preferences | — | — | 42s | 2026-10-08 18:47 |
| 156 | parcel_redelivery | haiku-replay@1008 | fail | slot | — | — | 39s | 2026-10-08 18:43 |
| 155 | parcel_redelivery | haiku-replay@1008 | fail | consent | — | — | 31s | 2026-10-08 18:41 |
| 154 | parcel_redelivery | haiku@replay-1008 | pass | done | 152,029 in / 10,002 out | $0.020 | 1m 36s | 2026-10-08 18:39 |
| 153 | parcel_redelivery | haiku-replay@1008 | active | — | — | — | — | 2026-10-08 16:20 |
| 152 | parcel_redelivery | haiku@replay-1008 | pass | done | 149,729 in / 10,351 out | $0.020 | 1m 44s | 2026-10-08 16:18 |
| 151 | parcel_redelivery | deepseek@local-1002 | pass | done | 284,463 in / 155,057 out | $0.256 | 13m 4s | 2026-10-02 17:21 |