Evaluating and Enhancing Machine Comprehension of Hybrid Youth Language in Kenyan Social Media

Abstract

Abstract

This study explored how Artificial Intelligence (AI) systems interpret Sheng, a dynamic hybrid youth language widely used in informal communication across Kenyan social media platforms. Although advancements in Natural Language Processing (NLP) have improved machine translation for well-resourced languages, hybrid and rapidly evolving forms such as Sheng remain underrepresented. As a result, AI system struggle to accurately interpret, leading to miscommunication and loss of culturally embedded meaning in digital spaces. Grounded in Sociolinguistics Theory, which views language as socially constructed and context-dependent, the study explored how meaning in Sheng is shaped by community practices and everyday online interactions. The study evaluated the accuracy of AI systems in interpreting and translating Sheng expressions. A mixed-methods research design was adopted, combining quantitative and qualitative approaches. The study analysed a purposive sample of 300 Sheng-language posts from Twitter, TikTok, and WhatsApp, alongside data from 60 Sheng-speaking youth selected through stratified sampling. AI system tools tested included Google Translate and ChatGPT. These were assessed based on their ability to interpret meaning, retain context and accurately translate Sheng expressions into English or Kiswahili. Quantitative data were analysed using descriptive statistics, while qualitative data were subjected to thematic analysis to identify recurring patterns, common errors, and contextual gaps in machine understanding. The findings revealed significant limitations in AI comprehension of Sheng, particularly in handling slang variation, code-switching, and cultural nuance. The study emphasized the need for more context-aware, linguistically inclusive NLP models, thereby contributing to the development of AI systems that enforce more effective multilingual digital communication.