openai/gpt-6-luna

openai family · run 15 November 2026 · each item asked 3 times

Overall score: 75.6%95% interval 73.9% to 77.2%
Rank: 1 of 1
Cost per run: $0.78this suite, judges included
Cost per correct: $0.000224
Consistency: 68%mean over tests

Scores by test

Test	Score	95% interval	Items	Consistency	Cost	Cost per correct
Lesson exercises	93.6%	91.1% to 95.7%	360	96%	$0.05	$5.39e-05
Word forms	80.5%	76.9% to 84.1%	360	86%	$0.17	$0.000196
Translation	68.0%	65.4% to 70.8%	360	44%	$0.12	$0.000166
Round trip	64.5%	61.7% to 67.4%	240	33%	$0.14	$0.000311
Tutor accuracy	71.3%	65.4% to 77.0%	180	79%	$0.28	$0.000737

Consistency is the share of items that got the same score in all 3 repeats. A low number means the model's answers vary from one ask to the next.

Tutor explanations

Tutor items also ask for a short explanation. Two judge models from other model families mark it. This is a secondary number: it is judge-marked and not human-validated.

Tutor accuracy

Headline score: 71.3%: found the error, fixed it and changed nothing else (rule-only, no judge).
Parts of the headline: Found 72.2% · fixed 71.5% · no over-correction 82.6%
Explanation score: 57.4% (95% interval 50.6% to 64.2%), judge-marked, not human-validated. An explanation passes only when both judges pass it.
Judge agreement: 93.2% of 527 judged answers
Unscored: 13 answers a judge could not mark, after 13 failed judge calls; they are left out, not scored 0.
Judges: deepseek/deepseek-v4.1-flash-1017, qwen/qwen3.8-flash

By language

Score, 95% interval and number of items (n) for each test.

Language	Lesson exercises	Word forms	Translation	Round trip	Tutor accuracy
Irish	90.0% 81.7–96.7 · n=60	87.2% 79.4–93.9 · n=60	76.9% 71.6–81.7 · n=60	78.2% 73.1–83.2 · n=40	73.3% 63.3–83.3 · n=60
Welsh	94.4% 88.3–98.9 · n=60	90.0% 83.9–95.0 · n=60	77.9% 73.2–82.6 · n=60	79.5% 73.8–84.5 · n=40	75.6% 65.6–85.0 · n=60
Scottish Gaelic	96.1% 91.7–99.4 · n=60	86.7% 77.8–94.4 · n=60	75.8% 69.9–81.7 · n=60	68.3% 62.2–74.6 · n=40	65.0% 54.4–75.6 · n=60
Manx	95.6% 90.0–99.4 · n=60	78.9% 68.3–87.8 · n=60	43.1% 38.4–47.9 · n=60	50.2% 44.7–55.6 · n=40	n/a
Breton	94.4% 88.9–98.9 · n=60	72.2% 62.2–81.7 · n=60	66.0% 59.7–72.1 · n=60	57.7% 51.3–64.9 · n=40	n/a
Cornish	91.1% 83.3–97.8 · n=60	67.8% 57.2–78.3 · n=60	68.4% 61.3–75.0 · n=60	52.9% 46.5–59.7 · n=40	n/a

By CEFR level

The CEFR level comes from the course content; items without one are listed as unspecified.

Score, 95% interval and number of items (n) for each test.

CEFR level	Lesson exercises	Word forms	Translation	Round trip	Tutor accuracy
A1	95.5% 91.7–98.1 · n=125	86.3% 80.8–91.3 · n=134	72.8% 67.6–78.2 · n=112	64.7% 56.2–73.5 · n=25	84.8% 71.2–97.0 · n=22
A2	90.9% 86.2–95.3 · n=135	75.7% 68.8–81.9 · n=140	62.0% 56.6–67.3 · n=91	66.6% 61.8–71.5 · n=88	74.1% 60.5–85.2 · n=27
B1	95.0% 90.0–99.1 · n=73	67.5% 55.6–79.5 · n=39	63.2% 56.5–69.8 · n=47	57.4% 52.4–62.5 · n=79	61.1% 48.4–73.8 · n=42
B2	94.4% 86.1–100.0 · n=24	70.8% 52.1–89.6 · n=16	62.9% 53.2–71.9 · n=30	67.0% 59.5–74.5 · n=27	65.2% 46.4–82.6 · n=23
C1	100.0% 100.0–100.0 · n=3	n/a	84.4% 76.5–91.6 · n=14	75.8% 66.9–84.7 · n=15	63.3% 36.7–86.7 · n=10
C2	n/a	n/a	78.7% 69.1–88.5 · n=6	86.2% 82.2–91.9 · n=6	50.0% 0.0–100.0 · n=2
Unspecified	n/a	97.8% 93.5–100.0 · n=31	69.6% 64.8–74.1 · n=60	n/a	77.2% 66.7–87.0 · n=54

Grammar topics

The topic is the grammar point or error type recorded on each item.

Lesson exercises

Strongest 5 topics.

Topic	Score	Items
adjectives	100.0%	5
chapter overview	100.0%	11
conversation	100.0%	6
food	100.0%	5
grammar point	100.0%	10

Weakest 5 topics.

Topic	Score	Items
syntax	66.7%	5
mutations	81.5%	9
vocabulary	85.7%	7
grammar	90.5%	21
unit focus	90.8%	51

Only topics with at least 5 items are ranked.

Word forms

Strongest 5 topics.

Topic	Score	Items
number personal	100.0%	9
article gender mutation	97.8%	31
copula	88.2%	31
number cardinal	87.8%	30
verb imperative	85.2%	27

Weakest 5 topics.

Topic	Score	Items
number counting	60.0%	5
verb past	66.7%	22
verb conditional	68.1%	23
verb imperfect	71.4%	14
noun plural	73.3%	30

Only topics with at least 5 items are ranked.

Translation

All topics, highest score first.

Topic	Score	Items
to english	75.0%	150
legal or administrative	69.6%	60
from english	60.4%	150

Tutor accuracy

All topics, highest score first.

Topic	Score	Items
orthography doubled letter	100.0%	7
clean	80.0%	30
guide	75.0%	20
grammar	71.4%	42
unit focus	66.7%	10
grammar point	63.9%	12
vocabulary	57.8%	15
application	48.5%	11

Only topics with at least 5 items are ranked.

Cost and prices

This suite: $0.78, of which $0.20 was judge spend
Whole run: $1.30 for both suites, against a cap of $3.00
Cost per correct answer: $0.000224: suite cost divided by the number of full-score answers
Run date: 15 November 2026
Prices for openai/gpt-6-luna: $0.1000 input, $0.50 output, $0.0100 cache read, $0.12 cache write per million tokens. Retrieved 15 November 2026 from https://openrouter.ai/api/v1/models.
Judge deepseek/deepseek-v4.1-flash-1017: $0.0200 input, $0.11 output, $0.0070 cache read per million tokens. Retrieved 15 November 2026 from OpenRouter.
Judge qwen/qwen3.8-flash: $0.15 input, $0.47 output, $0.0160 cache read per million tokens. Retrieved 15 November 2026 from OpenRouter.

Failure examples

The lowest-scoring items from the public samples, with the worst of the repeats. Private items are never shown. Model output is shown exactly as text.

Word forms

blas-morph:cornish:verb_imperative:kw-verb-bos:2p

Cornish · verb imperative · score 0.00

Prompt

Language: Cornish
Task: Give the imperative form of the verb for the person given. Write one word: the inflected verb form, with no pronoun, particle or auxiliary.
Verb: bos
Meaning: to be
Tense: imperative
Person: 2nd person plural (several people addressed)
Reply with the answer on one line, in this exact form:
ANSWER: <answer>

Expected

bedhewgh

Model output

ANSWER: bysowgh

blas-morph:manx:verb_conditional:gv-verb-cur:dependent:2s

Manx · verb conditional · score 0.00

Prompt

Language: Manx
Task: Give the conditional tense form of the verb: the dependent form (used after cha, nagh and the question particle). Write the verb form followed by the pronoun 'oo'.
Verb: cur
Meaning: to give, to put
Tense: conditional
Person: 2nd person singular (one person addressed) (oo)
Reply with the answer on one line, in this exact form:
ANSWER: <answer>

Expected

derragh oo

Model output

ANSWER: deragh oo

blas-morph:scots-gaelic:copula:cop-gd-empha-04

Scottish Gaelic · copula · score 0.00

Prompt

Language: Scottish Gaelic
Task: Fill in the blank with the copula word (or copula particle) that the sentence needs. Write only the missing word(s).
Sentence: ___ do Dhia a bhuineas slàinte.
English: Salvation is God's affair.
Reply with the answer on one line, in this exact form:
ANSWER: <answer>

Expected

'S ann

Model output

ANSWER: Is ann

Translation

blas-translation-scots-gaelic-gd-sent-0255-from_english

Scottish Gaelic · from english · score 0.25

Prompt

Translate this English sentence into Scottish Gaelic.
Reply with the Scottish Gaelic translation only, on one line, with no notes or explanation.

English: She used to drink beer.

Expected

B' àbhaist dhi leann òl.

Model output

Bhiodh i ag òl leann.

blas-translation-cornish-kw-sent-tat-1356-from_english

Cornish · from english · score 0.26

Prompt

Translate this English sentence into Cornish.
Reply with the Cornish translation only, on one line, with no notes or explanation.

English: Do you like dogs?

Expected

Yw da genes keun?

Model output

A gar'ta keun?

blas-translation-manx-gv-sent-parallel-15649-from_english

Manx · from english · score 0.41

Prompt

Translate this English sentence into Manx.
Reply with the Manx translation only, on one line, with no notes or explanation.

English: Have you made my bed?

Expected

Vel shiu er chiartaghey my lhiabbee?

Model output

Vel oo er n'yannoo my lhiabbee?

Round trip

blas-round_trip-cornish-kw-sent-tat-1830

Cornish · round trip · score 0.17

Prompt

Translate this Cornish sentence into English.
Reply with the English translation only, on one line, with no notes or explanation.

Cornish: Kas yw genev an traow na.

Expected

Kas yw genev an traow na.

Model output

My a gas an taklow-na.

Pivot text

I hate those things.

blas-round_trip-manx-gv-sent-parallel-2423

Manx · round trip · score 0.19

Prompt

Translate this Manx sentence into English.
Reply with the English translation only, on one line, with no notes or explanation.

Manx: Dreill eh smaght er eh hene.

Expected

Dreill eh smaght er eh hene.

Model output

Ren eh eh-hene y reirey.

Pivot text

He kept himself under control.

blas-round_trip-cornish-kw-sent-tat-0552

Cornish · round trip · score 0.28

Prompt

Translate this Cornish sentence into English.
Reply with the English translation only, on one line, with no notes or explanation.

Cornish: Mar pe dy’Sul hedhyw, ni a wrussa mos dhe byskessa.

Expected

Mar pe dy’Sul hedhyw, ni a wrussa mos dhe byskessa.

Model output

Mar bije hedhyw Dy'Sul, ni a yje pesketa.

Pivot text

If today were Sunday, we would go fishing.

Tutor accuracy

blas-tutor-welsh-exd-73d43ee1db

Welsh · grammar · score 0.00

Prompt

You are a Welsh language tutor checking a sentence written by a learner.
Language: Welsh
The sentence may contain one error, or it may already be correct.
If it has an error, change only that one error: give the exact erroneous substring and the replacement for it. If it is correct, set has_error to false, leave error_span and correction empty and repeat the sentence as corrected_sentence.
Sentence: Bues i'n gweithio fel athro am bum blynedd.
Reply with one JSON object and nothing else:
{"has_e…

Expected

Bues i'n gweithio fel athro am bum mlynedd.

Model output

{"has_error":false,"error_span":"","correction":"","corrected_sentence":"Bues i'n gweithio fel athro am bum blynedd.","explanation":"The sentence is grammatically correct."}

blas-tutor-welsh-exd-d6fa3b01fb

Welsh · grammar · score 0.00

Prompt

You are a Welsh language tutor checking a sentence written by a learner.
Language: Welsh
The sentence may contain one error, or it may already be correct.
If it has an error, change only that one error: give the exact erroneous substring and the replacement for it. If it is correct, set has_error to false, leave error_span and correction empty and repeat the sentence as corrected_sentence.
Sentence: Byddai yna adar y tu allan er i fyddai merlod na defaid.
Reply with one JSON object and nothing …

Expected

Byddai yna adar y tu allan er na fyddai merlod na defaid.

Model output

{"has_error":true,"error_span":"er i fyddai","correction":"er na fyddai","corrected_sentence":"Byddai yna adar y tu allan er na fyddai merlod na defaid.","explanation":"“Er na fyddai” means “although there would not be”; it correctly forms the negative clause with “na fyddai merlod na defaid.”"}

blas-tutor-welsh-exd-21c6c984df

Welsh · grammar · score 0.00

Prompt

You are a Welsh language tutor checking a sentence written by a learner.
Language: Welsh
The sentence may contain one error, or it may already be correct.
If it has an error, change only that one error: give the exact erroneous substring and the replacement for it. If it is correct, set has_error to false, leave error_span and correction empty and repeat the sentence as corrected_sentence.
Sentence: Cafodd ei anghofio gan pawb.
Reply with one JSON object and nothing else:
{"has_error": true|fal…

Expected

Cafodd ei anghofio gan bawb.

Model output

{"has_error":false,"error_span":"","correction":"","corrected_sentence":"Cafodd ei anghofio gan pawb.","explanation":"Mae’r frawddeg yn gywir: mae “Cafodd ei anghofio” yn ffurf oddefol, ac mae “gan bawb” yn gywir."}