01 — Why the language of the question matters
The same question, a different language, possibly a different answer
Where it bites
Malaria danger signs, oral rehydration, pregnancy warning signs, snakebite — cases where a missing sentence changes what a household does in the next hour.
And in paperwork
Health insurance, birth registration, the Ghana Card, a passport. A wrong document list in Ewe costs a wasted journey to a district office.
The standard applied
UNESCO's Recommendation on the Ethics of Artificial Intelligence names bias, fairness, human rights and proportionality as the areas to examine. Source
02 — The pilot, as a real case
Six systems, twelve questions, both languages
Every Ewe question came back in Ewe
Across 6 systems and 12 questions, each Ewe prompt received a reply that the language check read as Ewe rather than English. Refusal to use the language was not the failure mode here.
Ewe answers were consistently shorter
Measured in characters against the same system's English answer to the same question, the Ewe replies ranged from 44% to 82% of the English length. Shorter is not automatically worse, but it is where a missing warning or missing step would show up.
Care referral was named less often in Ewe by some systems
A keyword check for clinic, hospital, doctor, nurse or health worker in both languages found several systems naming a health worker in fewer Ewe answers than English answers to the same questions.
These are counts of length, reply language and keywords — measurable properties of the text. They are not accuracy, and this site does not present them as accuracy.
03 — What is still open
Everything that needs a human speaker
- Whether the Ewe answers are factually correct. A keyword and length check cannot see a wrong dose, a wrong age or a wrong office.
- Whether the Ewe reads naturally to a speaker in the Volta Region, or is a stiff translation of English phrasing.
- Whether any Ewe answer is unsafe — for example, advice that would delay someone taking a child with malaria danger signs to a clinic.
- Whether my own Ewe question wording is idiomatic. It was written by me and has not been checked by a fluent speaker.
Until qualified reviewers have worked through the transcripts, no accuracy or safety score is published. Reviewer scores appear on the pilot page only when the reviewer has submitted them and consented to publication.
04 — Take part