Testing platform for ZShot
| Template | Total Runs | Pass | Fail | Pass Rate | Avg Tokens In | Avg Tokens Out | Tokens In per Pass |
|---|---|---|---|---|---|---|---|
| benefits_hardship_claim | 32 | 16 | 16 | 50.0% | 898,848 | 19,789 | 1,235,917 |
| doom_destroy_a_barrel | 15 | 7 | 8 | 46.7% | 208,211 | 5,319 | 446,166 |
| parcel_redelivery | 281 | 116 | 165 | 41.3% | 181,211 | 15,199 | 412,846 |
| recipe_site_favorite_a_recipe | 16 | 8 | 8 | 50.0% | 15,315 | 609 | 15,315 |
Token figures count runs recorded with --llm-usage. Tokens in per pass is every metered run's input over the metered runs that passed.
| ID | Template | Label | Result | Furthest | Tokens | Est. Cost | Took | Created |
|---|---|---|---|---|---|---|---|---|
| 300 | recipe_site_favorite_a_recipe | opus@app049 | fail | — | — | — | 0s | 2026-10-16 16:56 |
| 299 | benefits_hardship_claim | tags-ar | pass | done | 315,220 in / 14,559 out | $0.642 | 7m 10s | 2026-10-16 11:32 |
| 298 | benefits_hardship_claim | tags-ja | pass | done | 245,703 in / 9,927 out | $0.479 | 4m 59s | 2026-10-16 11:27 |
| 297 | benefits_hardship_claim | tags-en | pass | done | 221,690 in / 10,856 out | $0.452 | 3m 50s | 2026-10-16 11:24 |
| 296 | parcel_redelivery | tags-ar | pass | done | 212,056 in / 12,718 out | $0.485 | 5m 55s | 2026-10-16 11:18 |
| 295 | parcel_redelivery | tags-ko | pass | done | 205,863 in / 9,539 out | $0.453 | 5m 12s | 2026-10-16 11:12 |
| 294 | parcel_redelivery | tags-ja | pass | done | 131,855 in / 6,555 out | $0.294 | 4m 3s | 2026-10-16 11:08 |
| 293 | parcel_redelivery | tags-en | pass | done | 189,332 in / 7,841 out | $0.413 | 4m 25s | 2026-10-16 11:04 |
| 292 | benefits_hardship_claim | tags-ar | fail | start | — | — | 0s | 2026-10-16 11:03 |
| 291 | benefits_hardship_claim | tags-ja | fail | start | — | — | 0s | 2026-10-16 11:03 |
| 290 | benefits_hardship_claim | tags-en | pass | done | 254,824 in / 10,838 out | $0.504 | 3m 13s | 2026-10-16 11:00 |
| 289 | parcel_redelivery | tags-ar | fail | consent | — | — | 0s | 2026-10-16 11:00 |
| 288 | parcel_redelivery | tags-ko | fail | consent | — | — | 0s | 2026-10-16 11:00 |
| 287 | parcel_redelivery | tags-ja | fail | consent | — | — | 0s | 2026-10-16 11:00 |
| 286 | parcel_redelivery | tags-en | pass | done | 126,219 in / 5,912 out | $0.278 | 2m 34s | 2026-10-16 10:57 |
| 285 | parcel_redelivery | docker-captions | active | — | — | — | — | 2026-10-16 02:18 |
| 284 | parcel_redelivery | docker-captions | active | — | — | — | — | 2026-10-16 02:17 |
| 283 | parcel_redelivery | docker-captions | active | — | — | — | — | 2026-10-16 02:16 |
| 282 | benefits_hardship_claim | captions-ar | pass | done | 177,095 in / 8,722 out | $0.357 | 2m 26s | 2026-10-16 01:46 |
| 281 | benefits_hardship_claim | captions-ja | pass | done | 283,184 in / 10,554 out | $0.552 | 3m 11s | 2026-10-16 01:43 |
| 280 | benefits_hardship_claim | captions-en | pass | done | 174,466 in / 7,905 out | $0.349 | 1m 46s | 2026-10-16 01:41 |
| 279 | parcel_redelivery | captions-ar | pass | done | 165,536 in / 6,549 out | $0.359 | 3m 6s | 2026-10-16 01:38 |
| 278 | parcel_redelivery | captions-ko | pass | done | 162,933 in / 7,106 out | $0.360 | 2m 42s | 2026-10-16 01:35 |
| 277 | parcel_redelivery | captions-ja | pass | done | 139,939 in / 4,974 out | $0.297 | 2m 40s | 2026-10-16 01:32 |
| 276 | parcel_redelivery | captions-en | pass | done | 140,129 in / 4,807 out | $0.297 | 1m 50s | 2026-10-16 01:30 |
| 275 | benefits_hardship_claim | captions-ar | fail | done | 293,608 in / 9,683 out | $0.555 | 3m 11s | 2026-10-16 01:26 |
| 274 | benefits_hardship_claim | captions-ja | pass | done | 268,919 in / 12,062 out | $0.544 | 4m 54s | 2026-10-16 01:21 |
| 273 | benefits_hardship_claim | captions-en | pass | done | 160,066 in / 7,670 out | $0.322 | 1m 59s | 2026-10-16 01:19 |
| 272 | parcel_redelivery | captions-ar | pass | done | 150,542 in / 5,788 out | $0.324 | 2m 26s | 2026-10-16 01:16 |
| 271 | parcel_redelivery | captions-ko | pass | done | 139,547 in / 5,714 out | $0.302 | 2m 15s | 2026-10-16 01:14 |
| 270 | parcel_redelivery | captions-ja | pass | done | 145,725 in / 5,085 out | $0.311 | 2m 11s | 2026-10-16 01:12 |
| 269 | parcel_redelivery | captions-en | pass | done | 137,661 in / 4,957 out | $0.293 | 2m 38s | 2026-10-16 01:09 |
| 268 | benefits_hardship_claim | captions-ar | pass | done | 234,475 in / 9,053 out | $0.449 | 2m 10s | 2026-10-16 01:03 |
| 267 | benefits_hardship_claim | captions-ja | pass | done | 182,284 in / 9,267 out | $0.376 | 2m 27s | 2026-10-16 01:00 |
| 266 | benefits_hardship_claim | captions-en | pass | done | 164,281 in / 7,825 out | $0.332 | 1m 52s | 2026-10-16 00:58 |
| 265 | parcel_redelivery | captions-ar | pass | done | 137,092 in / 6,037 out | $0.301 | 2m 5s | 2026-10-16 00:56 |
| 264 | parcel_redelivery | captions-ko | pass | done | 165,501 in / 6,519 out | $0.358 | 2m 26s | 2026-10-16 00:54 |
| 263 | parcel_redelivery | captions-ja | pass | done | 131,523 in / 5,604 out | $0.288 | 2m 6s | 2026-10-16 00:52 |
| 262 | parcel_redelivery | captions-en | pass | done | 138,968 in / 4,692 out | $0.296 | 2m 24s | 2026-10-16 00:49 |
| 261 | parcel_redelivery | sonnet@captions2-1015 | pass | done | 33,228 in / 1,096 out | $0.077 | 50s | 2026-10-15 22:27 |
| 260 | parcel_redelivery | sonnet@captions-1015 | fail | human | 29,669 in / 1,464 out | $0.068 | 1m 17s | 2026-10-15 22:25 |
| 259 | parcel_redelivery | scout-sonnet-opus@ts-off2-1015 | fail | done | 291,874 in / 11,845 out | $0.212 | 1m 18s | 2026-10-15 13:34 |
| 258 | parcel_redelivery | scout-sonnet-opus@ts-img2-1015 | fail | done | 153,772 in / 6,787 out | $0.093 | 57s | 2026-10-15 13:33 |
| 257 | parcel_redelivery | scout-sonnet-opus@ts-off2-1015 | pass | done | 376,858 in / 12,853 out | $0.390 | 1m 55s | 2026-10-15 13:32 |
| 256 | parcel_redelivery | scout-sonnet-opus@ts-img2-1015 | pass | done | 624,096 in / 16,335 out | $0.668 | 4m 6s | 2026-10-15 13:27 |
| 255 | parcel_redelivery | scout-sonnet-opus@ts-off-1015 | fail | confirm | 384,258 in / 13,667 out | $0.377 | 1m 46s | 2026-10-15 13:14 |
| 254 | parcel_redelivery | scout-sonnet-opus@ts-img-1015 | fail | confirm | 358,354 in / 9,522 out | $0.338 | 2m 26s | 2026-10-15 13:12 |
| 253 | parcel_redelivery | scout-sonnet-opus@ts-off-1015 | pass | done | 465,541 in / 14,933 out | $0.613 | 2m 59s | 2026-10-15 13:09 |
| 252 | parcel_redelivery | scout-sonnet-opus@ts-img-1015 | fail | confirm | 126,482 in / 5,552 out | $0.065 | 43s | 2026-10-15 13:08 |
| 251 | parcel_redelivery | opus@pointer-1015 | pass | done | 120,859 in / 3,714 out | $0.508 | 1m 56s | 2026-10-15 11:29 |