Publication Beyond Metrics: Evaluating LLMs’ Effectiveness in Culturally Nuanced, Low-Resource Real-World Scenarios Millicent Ochieng, Varun Gumma, Sunayana Sitaram, Jindong Wang, Vishrav Chaudhary, Keshet Ronen, Kalika Bali, Jacki O'Neill Association for Computational Linguistics (ACL 2025) | May 2025 Project
Publication Social by Nature: How Socio-tecture Shapes the Work of SMBs and Considerations for Reimagining Collaborative Human-AI Systems Elizabeth Ankrah, Kagonya Awori, Stephanie Nyairo, Mercy Muchai, Millicent Ochieng, Mark Kariuki, Gillian R Hayes, Jacki O'Neill CHI 2025 | April 2025 Project
Publication Towards Flood Extent Forecasting: Evaluating a Weather Foundation Model and U-Net for Flood Forecasting Samuel Chege Maina, Eric Wanjau Climate Change AI | April 2025 Project
Publication IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models David Ifeoluwa Adelani, Jessica Ojo, Israel Abebe Azime, Jian Yun Zhuang, Jesujoba O. Alabi, Xuanli He, Millicent Ochieng, Sara Hooker, Andiswa Bukula, En-Shiun Annie Lee, Chiamaka Chukwuneke, Happy Buzaaba, Blessing Sibanda, Godson Kalipe, Jonathan Mukiibi, Salomon Kabongo, Foutse Yuehgoh, Mmasibidi Setaka, Lolwethu Ndolela, Nkiruka Odu, Rooweither Mabuya, Shamsuddeen Hassan Muhammad, Salomey Osei, Sokhar Samb, Tadesse Kebede Guge, Tombekai Vangoni Sherman, Pontus Stenetorp NAACL 2025 | April 2025 Outstanding Paper Award at NAACL 2025 Project
Publication Dukawalla: Voice Interfaces for Small Businesses in Africa Elizabeth Ankrah, Stephanie Nyairo, Mercy Muchai, Kagonya Awori, Millicent Ochieng, Mark Kariuki, Jacki O'Neill ArXiv | November 2024 Project
Publication Towards Measuring and Modeling “Culture” in LLMs: A Survey Muhammad Farid Adilazuarda, Sagnik Mukherjee, Pradhyumna Lavania, Siddhant Singh, Ashutosh Dwivedi, Alham Fikri Aji, Jacki O'Neill, Ashutosh Modi, M. Choudhury EMNLP 2024 | November 2024
Publication Working with Generative AI: We need more African Voices, Real African Voices Najeeb G. Abdulhamid, Stephanie Nyairo, Jacki O'Neill November 2024 Project
Publication White Paper on AI and the Future of Work in Africa Jacki O'Neill, Vukosi Marivate, Barbara Glover, Winnie Karanu, Girmaw Abebe Tadesse, et.al. May 2024 Project
Publication METAL: Towards Multilingual Meta-Evaluation Rishav Hada, Varun Gumma, Mohamed Ahmed, Kalika Bali, Sunayana Sitaram ArXiv | April 2024, Vol abs/2404.01667: pp. 1051-1070
Publication Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks Sanchit Ahuja, Divyanshu Aggarwal, Varun Gumma, Ishaan Watts, Ashutosh Sathe, Millicent Ochieng, Rishav Hada, Prachi Jain, Mohamed Ahmed, Kalika Bali, Sunayana Sitaram North American Chapter of the Association for Computational Linguistics | March 2024 Project
