Testing platform for ZShot
| Template | Total Runs | Pass | Fail | Pass Rate | Avg Tokens In | Avg Tokens Out | Tokens In per Pass |
|---|---|---|---|---|---|---|---|
| benefits_hardship_claim | 32 | 16 | 16 | 50.0% | 898,848 | 19,789 | 1,235,917 |
| doom_destroy_a_barrel | 15 | 7 | 8 | 46.7% | 208,211 | 5,319 | 446,166 |
| parcel_redelivery | 281 | 116 | 165 | 41.3% | 181,211 | 15,199 | 412,846 |
| recipe_site_favorite_a_recipe | 16 | 8 | 8 | 50.0% | 15,315 | 609 | 15,315 |
Token figures count runs recorded with --llm-usage. Tokens in per pass is every metered run's input over the metered runs that passed.
| ID | Template | Label | Result | Furthest | Tokens | Est. Cost | Took | Created |
|---|---|---|---|---|---|---|---|---|
| 350 | parcel_redelivery | sonnet@titles | pass | done | 137,987 in / 5,674 out | $0.297 | 3m 9s | 2026-10-16 18:41 |
| 349 | benefits_hardship_claim | sonnet@titles | pass | done | 717,149 in / 19,351 out | $1.41 | 8m 26s | 2026-10-16 18:38 |
| 348 | parcel_redelivery | opus@titles | pass | done | 128,660 in / 3,999 out | $0.521 | 2m 45s | 2026-10-16 18:38 |
| 347 | benefits_hardship_claim | scout-clef@app-0.49 | fail | household | 5,534,567 in / 18,137 out | $2.49 | 18m 24s | 2026-10-16 18:00 |
| 346 | benefits_hardship_claim | scout-sonnet-opus@app-0.49 | fail | household | 5,709,474 in / 120,426 out | $3.00 | 7m 2s | 2026-10-16 17:53 |
| 345 | benefits_hardship_claim | deepseek@app-0.49 | fail | about_you | 696,547 in / 66,360 out | $0.244 | 14m 42s | 2026-10-16 17:38 |
| 344 | benefits_hardship_claim | luna@app-0.49 | fail | work | 1,564,572 in / 17,240 out | $0.146 | 8m 33s | 2026-10-16 17:30 |
| 343 | benefits_hardship_claim | gpt@app-0.49 | pass | done | 527,984 in / 2,276 out | $4.62 | 4m 5s | 2026-10-16 17:26 |
| 342 | parcel_redelivery | scout-clef@app-0.49 | pass | done | 162,842 in / 2,663 out | $0.203 | 1m 23s | 2026-10-16 17:25 |
| 341 | benefits_hardship_claim | haiku@app-0.49 | fail | about_you | 219,617 in / 18,867 out | $0.027 | 3m 41s | 2026-10-16 17:22 |
| 340 | parcel_redelivery | scout-sonnet-opus@app-0.49 | fail | preferences | 628,451 in / 16,779 out | $0.599 | 3m 16s | 2026-10-16 17:21 |
| 339 | parcel_redelivery | deepseek@app-0.49 | fail | done | 115,504 in / 24,356 out | $0.055 | 4m 19s | 2026-10-16 17:17 |
| 338 | parcel_redelivery | luna@app-0.49 | pass | done | 283,929 in / 6,862 out | $0.032 | 3m 0s | 2026-10-16 17:14 |
| 337 | parcel_redelivery | gpt@app-0.49 | pass | done | 168,122 in / 1,575 out | $1.76 | 2m 13s | 2026-10-16 17:12 |
| 336 | benefits_hardship_claim | sonnet@app-0.49 | pass | done | 1,012,852 in / 24,207 out | $2.00 | 11m 57s | 2026-10-16 17:10 |
| 335 | parcel_redelivery | haiku@app-0.49 | pass | done | 158,275 in / 13,191 out | $0.021 | 3m 37s | 2026-10-16 17:08 |
| 334 | recipe_site_favorite_a_recipe | scout-clef@app-0.49 | pass | — | 14,134 in / 0 out | $0.003 | 12s | 2026-10-16 17:02 |
| 333 | recipe_site_favorite_a_recipe | scout-sonnet-opus@app-0.49 | pass | — | 15,319 in / 1,392 out | $0.006 | 9s | 2026-10-16 17:02 |
| 332 | parcel_redelivery | sonnet@app-0.49 | pass | done | 281,568 in / 17,054 out | $0.680 | 6m 17s | 2026-10-16 17:02 |
| 331 | recipe_site_favorite_a_recipe | deepseek@app-0.49 | pass | — | 12,553 in / 534 out | $0.004 | 26s | 2026-10-16 17:02 |
| 330 | recipe_site_favorite_a_recipe | luna@app-0.49 | pass | — | 12,761 in / 302 out | $0.001 | 20s | 2026-10-16 17:01 |
| 329 | recipe_site_favorite_a_recipe | gpt@app-0.49 | pass | — | 12,730 in / 106 out | $0.133 | 20s | 2026-10-16 17:01 |
| 328 | recipe_site_favorite_a_recipe | haiku@app-0.49 | pass | — | 18,752 in / 1,406 out | $0.003 | 46s | 2026-10-16 17:00 |
| 327 | recipe_site_favorite_a_recipe | sonnet@app-0.49 | pass | — | 16,033 in / 519 out | $0.037 | 45s | 2026-10-16 16:59 |
| 326 | parcel_redelivery | opus@app-0.49 | pass | done | 144,140 in / 5,028 out | $0.601 | 3m 12s | 2026-10-16 16:59 |
| 325 | benefits_hardship_claim | opus@app-0.49 | pass | done | 816,100 in / 19,587 out | $3.16 | 11m 17s | 2026-10-16 16:59 |
| 324 | recipe_site_favorite_a_recipe | opus@app-0.49 | pass | — | 20,242 in / 617 out | $0.093 | 41s | 2026-10-16 16:59 |
| 323 | recipe_site_favorite_a_recipe | scout-clef@app049 | fail | — | — | — | 0s | 2026-10-16 16:56 |
| 322 | parcel_redelivery | scout-clef@app049 | fail | consent | — | — | 0s | 2026-10-16 16:56 |
| 321 | benefits_hardship_claim | scout-clef@app049 | fail | start | — | — | 0s | 2026-10-16 16:56 |
| 320 | parcel_redelivery | scout-sonnet-opus@app049 | fail | consent | — | — | 0s | 2026-10-16 16:56 |
| 319 | recipe_site_favorite_a_recipe | scout-sonnet-opus@app049 | fail | — | — | — | 0s | 2026-10-16 16:56 |
| 318 | benefits_hardship_claim | scout-sonnet-opus@app049 | fail | start | — | — | 0s | 2026-10-16 16:56 |
| 317 | parcel_redelivery | deepseek@app049 | fail | consent | — | — | 0s | 2026-10-16 16:56 |
| 316 | recipe_site_favorite_a_recipe | deepseek@app049 | fail | — | — | — | 0s | 2026-10-16 16:56 |
| 315 | benefits_hardship_claim | deepseek@app049 | fail | start | — | — | 0s | 2026-10-16 16:56 |
| 314 | parcel_redelivery | luna@app049 | fail | consent | — | — | 0s | 2026-10-16 16:56 |
| 313 | recipe_site_favorite_a_recipe | luna@app049 | fail | — | — | — | 0s | 2026-10-16 16:56 |
| 312 | benefits_hardship_claim | luna@app049 | fail | start | — | — | 0s | 2026-10-16 16:56 |
| 311 | parcel_redelivery | gpt@app049 | fail | consent | — | — | 0s | 2026-10-16 16:56 |
| 310 | recipe_site_favorite_a_recipe | gpt@app049 | fail | — | — | — | 0s | 2026-10-16 16:56 |
| 309 | benefits_hardship_claim | gpt@app049 | fail | start | — | — | 0s | 2026-10-16 16:56 |
| 308 | parcel_redelivery | haiku@app049 | fail | consent | — | — | 0s | 2026-10-16 16:56 |
| 307 | recipe_site_favorite_a_recipe | haiku@app049 | fail | — | — | — | 0s | 2026-10-16 16:56 |
| 306 | benefits_hardship_claim | haiku@app049 | fail | start | — | — | 0s | 2026-10-16 16:56 |
| 305 | parcel_redelivery | sonnet@app049 | fail | consent | — | — | 0s | 2026-10-16 16:56 |
| 304 | benefits_hardship_claim | sonnet@app049 | fail | start | — | — | 0s | 2026-10-16 16:56 |
| 303 | recipe_site_favorite_a_recipe | sonnet@app049 | fail | — | — | — | 0s | 2026-10-16 16:56 |
| 302 | benefits_hardship_claim | opus@app049 | fail | start | — | — | 0s | 2026-10-16 16:56 |
| 301 | parcel_redelivery | opus@app049 | fail | consent | — | — | 0s | 2026-10-16 16:56 |