Samuel Chege Maina
Author directory2026
Toward Cultural Alignment: Human-Centered Evaluation of Multimodal AI Stories Across Five African Communities
Millicent Ochieng | Felermino D. M. A. Ali | Elizabeth A. Ankrah | Najeeb Gambo Abdulhamid | Migisha Boyd | Stephanie Nyairo | Mercy Muchai | Samuel Chege Maina | Aditya Vashistha | Anja Thieme | Jacki O’Neill
Findings of the Association for Computational Linguistics: EMNLP 2026
Millicent Ochieng | Felermino D. M. A. Ali | Elizabeth A. Ankrah | Najeeb Gambo Abdulhamid | Migisha Boyd | Stephanie Nyairo | Mercy Muchai | Samuel Chege Maina | Aditya Vashistha | Anja Thieme | Jacki O’Neill
Findings of the Association for Computational Linguistics: EMNLP 2026
In this paper, we examine how well AI-generated multimodal stories align with the lived practices, relationships, language, values, and visual expectations of the communities they represent. We conduct a community-grounded mixed-methods evaluation with 19 culture representatives across five African communities, combining quantitative annotations with qualitative focus group discussions. We find that cultural alignment depends not simply on recognizable cultural markers, but on how those markers fit social, linguistic, procedural, and visual context. From these evaluations, we develop a taxonomy of cultural alignment comprising five broader cultural marker categories and eight recurring mechanisms of misalignment. We additionally evaluate five multimodal LLM judges to examine whether automated evaluation can approximate community-grounded judgments at scale. Judge reliability and score calibration vary substantially across communities, with no single judge performing consistently across all five settings. These findings motivate community-calibrated evaluation pipelines in which automated judges are validated against community judgments to determine where they can be trusted and where human review remains necessary.
Reasoning Beyond Labels: Measuring LLM Sentiment in Low-Resource, Culturally Nuanced Contexts
Millicent Ochieng | Anja Thieme | Ignatius Ezeani | Risa Ueno | Samuel Chege Maina | Keshet Ronen | Javier Gonzalez | Jacki O'Neill
Proceedings of the 7th Workshop on African Natural Language Processing (AfricaNLP 2026)
Millicent Ochieng | Anja Thieme | Ignatius Ezeani | Risa Ueno | Samuel Chege Maina | Keshet Ronen | Javier Gonzalez | Jacki O'Neill
Proceedings of the 7th Workshop on African Natural Language Processing (AfricaNLP 2026)
Sentiment analysis in low-resource, culturally nuanced contexts challenges conventional NLP approaches that assume fixed labels and universal affective expressions. We present a diagnostic framework that treats sentiment as a context-dependent, culturally embedded construct, and evaluate how large language models (LLMs) reason about sentiment in informal, code-mixed WhatsApp messages from Nairobi youth health groups. Using human-annotated data, sentiment-flipped counterfactuals, and rubric-based explanation evaluation, we probe LLM interpretability, robustness, and alignment with human reasoning. Framing our evaluation through a social science measurement lens, we operationalize LLM outputs as an instrument for measuring the abstract concept of sentiment. Our findings reveal significant variation in model reasoning quality, with top-tier LLMs demonstrating greater interpretive stability, while smaller open-weight models in our study show reduced stability under ambiguity or sentiment shifts. This work highlights the need for culturally sensitive, reasoning-aware AI evaluation in complex, real-world communication.