4 Moments That Made Me Realise AI Has an Indigenous Problem
The Text-to-Speech Disaster
"Halfway through, it started speaking French"
While practising my research presentation, I used a free AI text-to-speech tool to listen to my speech as I worked. I was curious how it would handle the te reo Māori and Portuguese words I'd woven throughout my presentation. I selected New Zealand as my region, but I was hopeful the Portuguese would be translated correctly.
It butchered the te reo words completely. The Portuguese was also incorrect. And halfway through, it randomly started speaking French.
Listening to this AI misinterpret languages that carry deep meaning and cultural significance, I found myself asking: Who was involved in building this translation tool? Who's doing the quality checks to make sure it's correct?
Because clearly, no one who actually speaks these languages was in the room when this system was designed.
Lost in Translation
“Beyond the Words: What Translation Misses”
After working closely with teams in Brazil and Colombia, I quickly learned that English wasn’t always the default language. Every day, I relied on Google Translate to bridge the language gap, translating between Portuguese and Spanish, and vice versa.
My teammates would assure me, “Yes, that’s correct,” and I’d do the same in return. But there was always this quiet doubt lingering in the background: Is it really right?
Was something missing, some nuance, some protocol, some feeling?
Even when the translation was “good enough,” I couldn’t help but feel uneasy as if I was skating over the top of their language instead of showing respect for the stories, traditions, and meanings behind every word. I started to realise that language isn’t just about getting information across; it’s about honouring the people and the context behind it.
The ChatGPT Hallucination
"Making up our words"
Curious, I once asked ChatGPT to explain a Māori proverb. It responded instantly with a beautiful, detailed explanation… of a proverb that doesn’t actually exist. Confident. Convincing. Completely fabricated.
This isn’t just a funny mistake. When AI systems are trained on limited, surface-level Indigenous data, they don’t just mistranslate, they hallucinate our culture. They invent knowledge, flatten nuance, and spread misinformation, all while sounding authoritative.
Imagine the impact: educators, policymakers, or learners trusting these outputs, unknowingly perpetuating errors and erasing the real richness of our traditions.
"AI-generated content issues: Tools like ChatGPT and DALL·E have produced culturally insensitive or nonsensical outputs when prompted with Indigenous languages or symbols." – Carlson & Frazee, 2024
This moment taught me: If we aren’t guiding the data and the design, AI will confidently rewrite our stories wrongly.
The Digital Language Trap
"Learn authentic Indigenous languages!"
Scrolling through the app store, I found language learning apps proudly advertising “authentic” Indigenous language courses. But dig deeper, and you see they were created without community involvement, filled with errors, and sometimes even sold under fake identities.
These apps profit from our heritage while teaching it wrong. They have flashier marketing than actual community-led language programs and none of the accountability.
"AI-generated ‘language learning’ resources featuring Indigenous languages are being sold online under fake identities, rife with mistranslations and lacking community verification." – Anishinabek News, 2024
It’s not just about getting a word wrong. It’s about who gets to profit, who gets to teach, and who gets to decide what “authentic” means. When our languages become a commodity, our stories are at risk of being lost, distorted, or stolen.