Teanga Bench

Results by language

How each model scores for each language: the mean of its per-test scores, every test counted equally.

Items and answers come from the named sources and have not been reviewed by people, so a few gold answers may be wrong. How the items are built

Average across 9 models

#

Averaged over 9 models, scores are highest for Welsh (85.1%) and lowest for Manx (65.2%).

Mean of each model's per-test scores for each language, averaged over models; highest first. Ranges per model are on each model page.

  1. Welsh
  2. Irish
  3. Scottish Gaelic
  4. Breton
  5. Cornish
  6. Manx
View as table
LanguageAverageLowest modelHighest model
Welsh85.1%Qwen3.8 Flash (69.9%)Claude Fable 5.2 (91.5%)
Irish82.2%Qwen3.8 Flash (69.1%)Claude Fable 5.2 (89.8%)
Scottish Gaelic80.6%Qwen3.8 Flash (71.2%)Claude Fable 5.2 (86.6%)
Breton74.3%Claude Haiku 5.5 (60.8%)Claude Fable 5.2 (83.4%)
Cornish73.9%Qwen3.8 Flash (49.6%)Claude Fable 5.2 (86.3%)
Manx65.2%Claude Haiku 5.5 (44.9%)Claude Fable 5.2 (74.8%)

Every model

#

Darker cells are higher. Select a model for its ranges and item counts.

ModelIrishWelshScottish GaelicManxBretonCornish
Claude Fable 5.289.8%91.5%86.6%74.8%83.4%86.3%
Gemini 4 Hydrogen87.3%89.4%86.1%74.3%80.9%83.1%
GPT-6 Astra86.6%89.2%83.8%72.3%82.4%82.9%
Gemini 4 Argon87.3%88.2%82.5%71.6%78.9%78.2%
DeepSeek V4.1 Flash 101781.3%86.7%81.8%68.6%77.2%78.9%
Kimi K3.1 Pro83.5%85.0%82.1%68.6%71.2%75.3%
GPT-6 Luna81.1%83.5%78.4%66.9%72.6%70.0%
Claude Haiku 5.573.6%82.7%73.2%44.9%60.8%60.5%
Qwen3.8 Flash69.1%69.9%71.2%45.0%61.5%49.6%

Test by test

#

Lesson exercises

ModelIrishWelshScottish GaelicManxBretonCornish
Claude Fable 5.297.2%98.9%100.0%96.7%98.3%100.0%
Gemini 4 Hydrogen93.3%100.0%100.0%97.8%95.6%97.2%
GPT-6 Astra93.3%96.7%98.3%95.6%95.6%100.0%
Gemini 4 Argon95.0%99.4%98.3%99.4%98.3%98.9%
DeepSeek V4.1 Flash 101792.2%96.1%98.3%93.9%96.1%96.7%
Kimi K3.1 Pro93.3%95.6%97.2%94.4%96.7%97.8%
GPT-6 Luna90.0%94.4%96.1%95.6%94.4%91.1%
Claude Haiku 5.587.2%91.7%90.6%79.4%88.3%87.8%
Qwen3.8 Flash88.9%90.6%94.4%87.8%93.3%88.3%

Word forms

ModelIrishWelshScottish GaelicManxBretonCornish
Claude Fable 5.296.7%97.8%96.7%86.1%97.8%89.4%
Gemini 4 Hydrogen92.2%95.0%91.7%88.3%95.0%84.4%
GPT-6 Astra91.7%100.0%91.7%87.2%97.8%86.7%
Gemini 4 Argon88.3%91.1%87.8%82.8%87.2%75.0%
DeepSeek V4.1 Flash 101787.2%96.1%91.1%80.0%85.6%82.2%
Kimi K3.1 Pro90.6%92.2%87.8%80.6%67.2%74.4%
GPT-6 Luna87.2%90.0%86.7%78.9%72.2%67.8%
Claude Haiku 5.575.6%90.0%73.3%42.2%53.9%56.7%
Qwen3.8 Flash62.8%53.9%65.6%46.1%44.4%30.6%

Translation

ModelIrishWelshScottish GaelicManxBretonCornish
Claude Fable 5.282.0%83.8%77.3%52.5%71.9%83.0%
Gemini 4 Hydrogen81.4%79.1%76.0%49.7%69.3%79.1%
GPT-6 Astra80.8%80.8%77.0%49.6%71.6%78.9%
Gemini 4 Argon81.5%79.8%76.6%48.9%69.5%75.3%
DeepSeek V4.1 Flash 101776.9%76.3%76.4%46.2%65.6%71.2%
Kimi K3.1 Pro78.0%78.9%77.5%46.2%64.3%71.6%
GPT-6 Luna76.9%77.9%75.8%43.1%66.0%68.4%
Claude Haiku 5.569.7%77.1%70.5%26.5%52.9%50.3%
Qwen3.8 Flash64.9%70.7%74.9%24.5%58.3%48.3%

Round trip

ModelIrishWelshScottish GaelicManxBretonCornish
Claude Fable 5.282.0%83.6%77.8%64.1%65.7%72.9%
Gemini 4 Hydrogen82.1%83.6%74.9%61.5%63.9%71.8%
GPT-6 Astra77.7%79.6%70.8%56.8%64.5%66.1%
Gemini 4 Argon82.8%82.4%74.9%55.3%60.7%63.8%
DeepSeek V4.1 Flash 101774.1%81.0%74.8%54.3%61.4%65.3%
Kimi K3.1 Pro78.6%76.9%69.3%53.2%56.6%57.5%
GPT-6 Luna78.2%79.5%68.3%50.2%57.7%52.9%
Claude Haiku 5.568.9%83.0%69.5%31.6%47.9%47.2%
Qwen3.8 Flash65.5%75.3%69.2%21.7%50.2%31.2%

Tutor accuracy

ModelIrishWelshScottish Gaelic
Claude Fable 5.291.1%93.3%81.1%
Gemini 4 Hydrogen87.2%89.4%87.8%
GPT-6 Astra89.4%88.9%81.1%
Gemini 4 Argon88.9%88.3%75.0%
DeepSeek V4.1 Flash 101776.1%83.9%68.3%
Kimi K3.1 Pro77.2%81.7%78.9%
GPT-6 Luna73.3%75.6%65.0%
Claude Haiku 5.566.7%71.7%62.2%
Qwen3.8 Flash63.3%58.9%51.7%