Models
9 models ranked. Select a model for every score, breakdown and cost.
Items and answers come from the named sources and have not been reviewed by people, so a few gold answers may be wrong. How the items are built
- #1Claude Fable 5.2Anthropic ·
anthropic/claude-fable-5.286.1%95% range 84.9–87.3%$31.79 per run$8.14 per 1,000 correct - #2Gemini 4 HydrogenGoogle ·
google/gemini-4-hydrogen84.4%95% range 83.1–85.6%$43.89 per run$11.49 per 1,000 correct - #3GPT-6 AstraOpenAI ·
openai/gpt-6-astra83.6%95% range 82.3–84.9%$26.95 per run$7.10 per 1,000 correct - #4Gemini 4 ArgonGoogle ·
google/gemini-4-argon81.9%95% range 80.5–83.3%$3.10 per run$0.83 per 1,000 correct - #5DeepSeek V4.1 Flash 1017DeepSeek ·
deepseek/deepseek-v4.1-flash-101779.2%95% range 77.7–80.7%$1.04 per run$0.29 per 1,000 correct - #6Kimi K3.1 ProMoonshot AI ·
moonshotai/kimi-k3.1-pro78.4%95% range 76.9–79.9%$36.20 per run$10.14 per 1,000 correct - #7GPT-6 LunaOpenAI ·
openai/gpt-6-luna75.6%95% range 73.9–77.2%$0.78 per run$0.22 per 1,000 correct - #8Claude Haiku 5.5Anthropic ·
anthropic/claude-haiku-5.567.1%95% range 65.3–68.9%$1.23 per run$0.40 per 1,000 correct - #9Qwen3.8 FlashQwen ·
qwen/qwen3.8-flash61.6%95% range 59.8–63.4%$10.45 per run$3.70 per 1,000 correct
Model profiles
#Claude Fable 5.2 has the highest score in 5 of 5 tests, but in every test the top score is tied with at least one other model (95% ranges overlap).
One axis per test, with the same scale on every profile: 0% at the centre to 100% at the outer ring. Solid accent: this model; faint dashed polygon: median model score on each test; faint dotted outer polygon: best score on each test (not necessarily one model). Vertex labels show this model’s scores. Compare scores, not polygon area; no confidence intervals are shown. Select a profile to open the model page; ordered by suite rank.
View as table
| Model | Lesson exercises | Word forms | Translation | Round trip | Tutor accuracy |
|---|---|---|---|---|---|
| Claude Fable 5.2 | 98.5% | 94.1% | 75.1% | 74.3% | 88.5% |
| Gemini 4 Hydrogen | 97.3% | 91.1% | 72.4% | 73.0% | 88.1% |
| GPT-6 Astra | 96.6% | 92.5% | 73.1% | 69.3% | 86.5% |
| Gemini 4 Argon | 98.2% | 85.4% | 71.9% | 70.0% | 84.1% |
| DeepSeek V4.1 Flash 1017 | 95.6% | 87.0% | 68.8% | 68.5% | 76.1% |
| Kimi K3.1 Pro | 95.8% | 82.1% | 69.4% | 65.3% | 79.3% |
| GPT-6 Luna | 93.6% | 80.5% | 68.0% | 64.5% | 71.3% |
| Claude Haiku 5.5 | 87.5% | 65.3% | 57.9% | 58.0% | 66.9% |
| Qwen3.8 Flash | 90.6% | 50.6% | 56.9% | 52.2% | 58.0% |
Score against cost
#GPT-6 Astra is the cheapest model tied with the leader (95% ranges overlap): 83.6% for $26.95, against 86.1% for $31.79 from Claude Fable 5.2.
Each circle is a model, numbered by rank. Higher is better; further left is cheaper. The vertical line is the 95% range. Cost is one run of this suite, judges included.
- Claude Fable 5.2 86.1% · $31.79
- Gemini 4 Hydrogen 84.4% · $43.89
- GPT-6 Astra 83.6% · $26.95
- Gemini 4 Argon 81.9% · $3.10
- DeepSeek V4.1 Flash 1017 79.2% · $1.04
- Kimi K3.1 Pro 78.4% · $36.20
- GPT-6 Luna 75.6% · $0.78
- Claude Haiku 5.5 67.1% · $1.23
- Qwen3.8 Flash 61.6% · $10.45
View as table
| Model | Rank | Score | 95% range | Cost per run |
|---|---|---|---|---|
| Claude Fable 5.2 | 1 | 86.1% | 84.9–87.3% | $31.79 |
| Gemini 4 Hydrogen | 2 | 84.4% | 83.1–85.6% | $43.89 |
| GPT-6 Astra | 3 | 83.6% | 82.3–84.9% | $26.95 |
| Gemini 4 Argon | 4 | 81.9% | 80.5–83.3% | $3.10 |
| DeepSeek V4.1 Flash 1017 | 5 | 79.2% | 77.7–80.7% | $1.04 |
| Kimi K3.1 Pro | 6 | 78.4% | 76.9–79.9% | $36.20 |
| GPT-6 Luna | 7 | 75.6% | 73.9–77.2% | $0.78 |
| Claude Haiku 5.5 | 8 | 67.1% | 65.3–68.9% | $1.23 |
| Qwen3.8 Flash | 9 | 61.6% | 59.8–63.4% | $10.45 |
Cost per correct answer, every model
Data: this leaderboard as JSON · every results file on GitHub