Workout-note model matrix

OpenRouter bench envelope (temperature 0, seed, json_schema name=prescription_rows, strict:false, usage.include). Production interpretPrescription is Base44 InvokeLLM({prompt, response_json_schema}) with model omitted, then sane() drops declined rows. Same RULES string; different envelope.

extract SHA 73860e2c00f5dbf3d74270fe5d0264b7f77b6d0f · ledger docs/benchmarks/2026-09-11-matrix-runs.ndjson · scored 1822 · errors 2 · ledger spend $1.7727 · generated 2026-09-11T17:41:03.446Z

Read this first

• Expected vs actual is a field table (green / blue). Unasserted keys (e.g. rpe, distance_m) are omitted — they are not “expected null”.
• Accuracy uses the last terminal: true success per (model, corpus_id, seed). HTTP 429s and parse_error are error rows, not hits or misses. After that filter n=1822 (not 1824).
• SCHEMA cannot emit distance_m or duration_seconds. Cardio lines are still graded as sets/reps (often 1×1).
• D-band EN is the reconstructed 696-char CrossFit transcript, not Sebastian’s 977-char dictation.
• deepseek / mistral often miss the rows[] envelope. google/gemini-2.5-flash ran 1 seed; others 3.
• Pass ≥70 in the offline HTML is a display threshold, not a product rule.

Recommendation (rescored after dropping distance_m + parse_error)

• Primary (interactive): google/gemini-2.5-flash
• Band D: google/gemini-3-flash-preview
• Honesty (0 invented, 0 weight_zero, accuracy ≥80, fastest p50): openai/gpt-5.6-luna · slower peer x-ai/grok-4.5

Per-model × band

llms.txt for a Base44 builder · every context page