Teanga Bench

Gemini 4 Hydrogen

Google · google/gemini-4-hydrogen · run 16 November 2026 · each item asked 3 times

Overall score
84.4%95% range 83.1–85.6%
Rank
2 of 9
Cost per run
$43.89this suite, judges included
Per 1,000 correct
$11.49cost of 1,000 full-score answers
Consistency
77.7%same score in all 3 asks, mean over tests

Items and answers come from the named sources and have not been reviewed by people, so a few gold answers may be wrong. How the items are built

Scores by test

#

Gemini 4 Hydrogen places best in Tutor accuracy (2nd of 9, tied with the top) and worst in Word forms (3rd of 9, 3.0 points below the top but tied with it); tied means the 95% ranges overlap.

Score on each test. Right: the score and its place among the ranked models. Select a test to compare every model.

  • This model, with its 95% range
  • Each other ranked model
  1. Lesson exercisesClaude Fable 5.2: 98.5%GPT-6 Astra: 96.6%Gemini 4 Argon: 98.2%DeepSeek V4.1 Flash 1017: 95.6%Kimi K3.1 Pro: 95.8%GPT-6 Luna: 93.6%Claude Haiku 5.5: 87.5%Qwen3.8 Flash: 90.6%Gemini 4 Hydrogen: 97.3%, 95% range 95.6% to 98.8%, 3rd of 9
  2. Word formsClaude Fable 5.2: 94.1%GPT-6 Astra: 92.5%Gemini 4 Argon: 85.4%DeepSeek V4.1 Flash 1017: 87.0%Kimi K3.1 Pro: 82.1%GPT-6 Luna: 80.5%Claude Haiku 5.5: 65.3%Qwen3.8 Flash: 50.6%Gemini 4 Hydrogen: 91.1%, 95% range 88.3% to 93.6%, 3rd of 9
  3. TranslationClaude Fable 5.2: 75.1%GPT-6 Astra: 73.1%Gemini 4 Argon: 71.9%DeepSeek V4.1 Flash 1017: 68.8%Kimi K3.1 Pro: 69.4%GPT-6 Luna: 68.0%Claude Haiku 5.5: 57.9%Qwen3.8 Flash: 56.9%Gemini 4 Hydrogen: 72.4%, 95% range 69.9% to 75.0%, 3rd of 9
  4. Round tripClaude Fable 5.2: 74.3%GPT-6 Astra: 69.3%Gemini 4 Argon: 70.0%DeepSeek V4.1 Flash 1017: 68.5%Kimi K3.1 Pro: 65.3%GPT-6 Luna: 64.5%Claude Haiku 5.5: 58.0%Qwen3.8 Flash: 52.2%Gemini 4 Hydrogen: 73.0%, 95% range 70.4% to 75.6%, 2nd of 9
  5. Tutor accuracyClaude Fable 5.2: 88.5%GPT-6 Astra: 86.5%Gemini 4 Argon: 84.1%DeepSeek V4.1 Flash 1017: 76.1%Kimi K3.1 Pro: 79.3%GPT-6 Luna: 71.3%Claude Haiku 5.5: 66.9%Qwen3.8 Flash: 58.0%Gemini 4 Hydrogen: 88.1%, 95% range 83.5% to 92.2%, 2nd of 9
View as table
TestScore and 95% rangePlaceHighestMedian model
Lesson exercises97.3% 95.6–98.8%3rd of 9Claude Fable 5.2 (98.5%)95.8%
Word forms91.1% 88.3–93.6%3rd of 9Claude Fable 5.2 (94.1%)85.4%
Translation72.4% 69.9–75.0%3rd of 9Claude Fable 5.2 (75.1%)69.4%
Round trip73.0% 70.4–75.6%2nd of 9Claude Fable 5.2 (74.3%)68.5%
Tutor accuracy88.1% 83.5–92.2%2nd of 9Claude Fable 5.2 (88.5%)79.3%

Gemini 4 Hydrogen places best in Tutor accuracy (2nd of 9, tied with the top) and worst in Word forms (3rd of 9, 3.0 points below the top but tied with it); tied means the 95% ranges overlap.

One axis per test, with the same scale on every profile: 0% at the centre to 100% at the outer ring. Solid accent: this model; faint dashed polygon: median model score on each test; faint dotted outer polygon: best score on each test (not necessarily one model). Vertex labels show this model’s scores. Compare scores, not polygon area; no confidence intervals are shown.

Gemini 4 Hydrogen test profileScale: 0% at the centre to 100% at the outer ring. Lesson exercises: 97.3%, median 95.8%, best 98.5%; Word forms: 91.1%, median 85.4%, best 94.1%; Translation: 72.4%, median 69.4%, best 75.1%; Round trip: 73.0%, median 68.5%, best 74.3%; Tutor accuracy: 88.1%, median 79.3%, best 88.5%97.3%Lessonexercises91.1%Word forms72.4%Translation73.0%Round trip88.1%Tutor accuracy
View as table
TestThis modelMedianBest
Lesson exercises97.3%95.8%98.5%
Word forms91.1%85.4%94.1%
Translation72.4%69.4%75.1%
Round trip73.0%68.5%74.3%
Tutor accuracy88.1%79.3%88.5%

Items, consistency and cost by test

Consistency is the share of items that got the same score in every repeat ask.

TestScore95% rangeItemsConsistencyCostPer 1,000 correct
Lesson exercises97.3%95.6–98.8%36099.2%$3.42$3.25
Word forms91.1%88.3–93.6%36092.2%$11.14$11.32
Translation72.4%69.9–75.0%36057.5%$10.66$13.63
Round trip73.0%70.4–75.6%24045.0%$13.72$26.12
Tutor accuracy88.1%83.5–92.2%18094.4%$4.95$10.39

By language

Score, 95% range and number of items (n) for each test.

LanguageLesson exercisesWord formsTranslationRound tripTutor accuracy
Irish93.3% 86.7–98.3% · n=6092.2% 86.1–97.8% · n=6081.4% 76.6–86.0% · n=6082.1% 77.0–86.9% · n=4087.2% 78.9–94.4% · n=60
Welsh100.0% 100.0–100.0% · n=6095.0% 89.4–99.4% · n=6079.1% 74.5–83.5% · n=6083.6% 78.9–88.3% · n=4089.4% 81.7–96.7% · n=60
Scottish Gaelic100.0% 100.0–100.0% · n=6091.7% 84.4–97.8% · n=6076.0% 70.4–81.6% · n=6074.9% 69.8–80.1% · n=4087.8% 79.4–95.0% · n=60
Manx97.8% 93.9–100.0% · n=6088.3% 81.1–95.0% · n=6049.7% 44.1–55.9% · n=6061.5% 55.8–67.0% · n=40n/a
Breton95.6% 90.0–100.0% · n=6095.0% 89.4–99.4% · n=6069.3% 63.2–75.1% · n=6063.9% 57.7–70.9% · n=40n/a
Cornish97.2% 92.8–100.0% · n=6084.4% 76.1–92.2% · n=6079.1% 73.4–85.1% · n=6071.8% 64.4–78.7% · n=40n/a

By CEFR level

The CEFR level comes from the course content; items without one are listed as unspecified.

Score, 95% range and number of items (n) for each test.

CEFR levelLesson exercisesWord formsTranslationRound tripTutor accuracy
A198.7% 96.5–100.0% · n=12592.0% 87.6–96.0% · n=13476.5% 71.6–81.3% · n=11269.2% 60.4–78.0% · n=2597.0% 90.9–100.0% · n=22
A295.6% 91.9–98.5% · n=13588.6% 83.8–92.9% · n=14069.2% 64.1–74.0% · n=9177.0% 72.6–81.4% · n=8888.9% 76.5–97.6% · n=27
B197.3% 93.6–100.0% · n=7394.0% 87.2–99.1% · n=3964.9% 57.6–71.7% · n=4767.4% 63.1–71.5% · n=7983.3% 72.2–92.9% · n=42
B2100.0% 100.0–100.0% · n=2487.5% 72.9–97.9% · n=1666.1% 56.4–75.0% · n=3070.9% 63.7–77.8% · n=2788.4% 73.9–100.0% · n=23
C1100.0% 100.0–100.0% · n=3n/a84.4% 77.5–90.6% · n=1479.2% 70.6–87.7% · n=1583.3% 60.0–100.0% · n=10
C2n/an/a80.5% 74.5–88.8% · n=695.9% 89.4–100.0% · n=666.7% 33.3–100.0% · n=2
Unspecifiedn/a96.8% 90.3–100.0% · n=3175.3% 70.4–80.1% · n=60n/a89.5% 81.5–96.3% · n=54

Grammar topics

The topic is the grammar point or error type recorded on each item.

Lesson exercises

Strongest 5 topics.

TopicScoreItems
adjectives100.0%5
chapter overview100.0%11
daily life100.0%5
food100.0%5
grammar point100.0%10

Weakest 5 topics.

TopicScoreItems
conversation83.3%6
vocabulary85.7%7
narrative grammar93.3%5
grammar95.2%21
unit focus96.7%51

Only topics with at least 5 items are ranked.

Word forms

Strongest 5 topics.

TopicScoreItems
verb pluperfect100.0%5
verb preterite100.0%5
article gender mutation96.8%31
number cardinal96.7%30
verb conditional95.7%23

Weakest 5 topics.

TopicScoreItems
number counting60.0%5
noun plural74.4%30
noun genitive85.2%9
copula87.1%31
number personal88.9%9

Only topics with at least 5 items are ranked.

Translation

All topics, highest score first.

TopicScoreItems
legal or administrative75.3%60
to english75.1%150
from english68.6%150

Tutor accuracy

All topics, highest score first.

TopicScoreItems
orthography doubled letter100.0%7
unit focus100.0%10
guide90.0%20
grammar88.1%42
no error86.7%30
application81.8%11
grammar point80.6%12
vocabulary77.8%15

Only topics with at least 5 items are ranked.

Tutor explanations

judge-marked, secondary

Tutor items also ask for a short explanation. Two judge models from other model families mark it. This is a secondary number: it is judge-marked and not human-validated.

Tutor accuracy

Headline score
88.1%: found the error, fixed it and changed nothing else (rule-only, no judge).
Parts of the headline
Found 88.3% · fixed 88.1% · no over-correction 91.3%
Explanation score
76.4% (95% range 70.4–82.0%), judge-marked, not human-validated. An explanation passes only when both judges pass it.
Judge agreement
95.1% of 511 judged answers
Unscored
29 answers a judge could not mark, after 29 failed judge calls; they are left out, not scored 0.
Judges
deepseek/deepseek-v4.1-flash-1017, qwen/qwen3.8-flash

Cost and prices

This suite
$43.89, of which $0.29 was judge spend
Whole run
$51.05 for both suites, against a cap of $65.00
Cost per correct answer
$0.01 ($11.49 per 1,000): suite cost divided by the number of full-score answers
Run date
16 November 2026
Prices for google/gemini-4-hydrogen
$1.50 input, $7.50 output, $0.0750 cache read, $0.0833 cache write per million tokens. Retrieved 16 November 2026 from https://openrouter.ai/api/v1/models.
Judge deepseek/deepseek-v4.1-flash-1017
$0.15 input, $0.60 output, $0.0030 cache read per million tokens. Retrieved 16 November 2026 from OpenRouter.
Judge qwen/qwen3.8-flash
$0.15 input, $0.47 output, $0.0160 cache read per million tokens. Retrieved 16 November 2026 from OpenRouter.

Failure examples

11

The lowest-scoring items from the public samples, with the worst of the repeats. Private items are never shown. Model output is shown exactly as text.

Word forms

blas-morph:manx:number_ordinal:gv-num-12

Manx · number ordinal · score 0.00

Prompt

Language: Manx
Task: Give the ordinal numeral for the number below: the ordinal ('first', 'second', ...), including the definite article yn as it stands before the ordinal.
Number: 12 (twelve)
Reply with the answer on one line, in this exact form:
ANSWER: <answer>

Expected

yn nah jeig

Model output

ANSWER: yn nah yeig

blas-morph:scots-gaelic:copula:cop-gd-empha-04

Scottish Gaelic · copula · score 0.00

Prompt

Language: Scottish Gaelic
Task: Fill in the blank with the copula word (or copula particle) that the sentence needs. Write only the missing word(s).
Sentence: ___ do Dhia a bhuineas slàinte.
English: Salvation is God's affair.
Reply with the answer on one line, in this exact form:
ANSWER: <answer>

Expected

'S ann

Model output

ANSWER: Is

blas-morph:cornish:verb_imperative:kw-verb-bos:2p

Cornish · verb imperative · score 0.00

Prompt

Language: Cornish
Task: Give the imperative form of the verb for the person given. Write one word: the inflected verb form, with no pronoun, particle or auxiliary.
Verb: bos
Meaning: to be
Tense: imperative
Person: 2nd person plural (several people addressed)
Reply with the answer on one line, in this exact form:
ANSWER: <answer>

Expected

bedhewgh

Model output

ANSWER: bydhewgh

Translation

blas-translation-scots-gaelic-gd-sent-0223-from_english

Scottish Gaelic · from english · score 0.15

Prompt

Translate this English sentence into Scottish Gaelic.
Reply with the Scottish Gaelic translation only, on one line, with no notes or explanation.

English: He is a good carpenter.

Expected

Tha e na dheagh shaor.

Model output

'S e saor math a th' ann.

blas-translation-manx-gv-sent-parallel-15649-from_english

Manx · from english · score 0.41

Prompt

Translate this English sentence into Manx.
Reply with the Manx translation only, on one line, with no notes or explanation.

English: Have you made my bed?

Expected

Vel shiu er chiartaghey my lhiabbee?

Model output

Vel oo er n'yannoo my lhiabbee?

blas-translation-breton-br-sent-0397-from_english

Breton · from english · score 0.44

Prompt

Translate this English sentence into Breton.
Reply with the Breton translation only, on one line, with no notes or explanation.

English: I'm from Australia.

Expected

Dont a ran eus Aostralia.

Model output

Eus Aostralia on.

Round trip

blas-round_trip-manx-gv-sent-parallel-6412

Manx · round trip · score 0.21

Prompt

Translate this Manx sentence into English.
Reply with the English translation only, on one line, with no notes or explanation.

Manx: Nee eh costal queig keead stiagh as magh.

Expected

Nee eh costal queig keead stiagh as magh.

Model output

Cosnee eh queig cheead, ny smoo ny ny sloo.

Pivot text

It will cost five hundred, more or less.

blas-round_trip-cornish-kw-sent-tat-1830

Cornish · round trip · score 0.26

Prompt

Translate this Cornish sentence into English.
Reply with the English translation only, on one line, with no notes or explanation.

Cornish: Kas yw genev an traow na.

Expected

Kas yw genev an traow na.

Model output

My a gas an taklow na.

Pivot text

I hate those things.

blas-round_trip-manx-gv-sent-parallel-4693

Manx · round trip · score 0.52

Prompt

Translate this Manx sentence into English.
Reply with the English translation only, on one line, with no notes or explanation.

Manx: Va mee sheiltyn dy darragh eh.

Expected

Va mee sheiltyn dy darragh eh.

Model output

Smooinee mee dy darragh eh.

Pivot text

I thought he would come.

Tutor accuracy

blas-tutor-welsh-exd-facf492c47

Welsh · grammar · score 0.00

Prompt

You are a Welsh language tutor checking a sentence written by a learner.
Language: Welsh
The sentence may contain one error, or it may already be correct.
If it has an error, change only that one error: give the exact erroneous substring and the replacement for it. If it is correct, set has_error to false, leave error_span and correction empty and repeat the sentence as corrected_sentence.
Sentence: y llyfr mae ar y bwrdd
Reply with one JSON object and nothing else:
{"has_error": true|false, "e…

Expected

y llyfr sydd ar y bwrdd

Model output

{"has_error": true, "error_span": "y llyfr mae", "correction": "mae'r llyfr", "corrected_sentence": "mae'r llyfr ar y bwrdd", "explanation": "In Welsh, a normal declarative sentence follows Verb-Subject-Object (VSO) word order, so the verb 'mae' must come before the subject 'y llyfr'. When 'mae' precedes 'y', the article contracts to ''r', giving 'mae'r llyfr ar y bwrdd'."}

blas-tutor-welsh-exd-d6fa3b01fb

Welsh · grammar · score 0.00

Prompt

You are a Welsh language tutor checking a sentence written by a learner.
Language: Welsh
The sentence may contain one error, or it may already be correct.
If it has an error, change only that one error: give the exact erroneous substring and the replacement for it. If it is correct, set has_error to false, leave error_span and correction empty and repeat the sentence as corrected_sentence.
Sentence: Byddai yna adar y tu allan er i fyddai merlod na defaid.
Reply with one JSON object and nothing …

Expected

Byddai yna adar y tu allan er na fyddai merlod na defaid.

Model output

{"has_error": true, "error_span": "er i", "correction": "er na", "corrected_sentence": "Byddai yna adar y tu allan er na fyddai merlod na defaid.", "explanation": "After 'er' (although), a negative finite verb like 'fyddai' requires the negative particle 'na' (meaning 'not'), rather than 'i' (which is used with verb-nouns). This also matches the negative conjunction 'na' in 'merlod na defaid'."}