Ruiwen Guan
Author directory2026
Beyond Factual QA: Mentorship-Oriented Question Answering over Long-Form Multilingual Content
Parth Bhalerao | Ruiwen Guan | Diola Dsouza | Oana Ignat
Findings of the Association for Computational Linguistics: EMNLP 2026
Parth Bhalerao | Ruiwen Guan | Diola Dsouza | Oana Ignat
Findings of the Association for Computational Linguistics: EMNLP 2026
Question answering systems are usually evaluated for factual correctness, but many real world settings such as education and career guidance require mentorship responses that offer reflection, context, and guidance. We introduce MENTORQA, the first multilingual dataset and evaluation framework for mentorship focused question answering from long form videos, with nearly 9,000 QA pairs from 180 hours of content across four languages. We define evaluation dimensions that measure fluency, clarity, alignment, and mentorship value, and use them to compare Single Agent, Dual Agent, RAG, and Multi Agent QA pipelines under controlled conditions. Multi Agent pipelines produce the strongest mentorship responses, with larger gains for complex topics and lower resource languages. We also study LLM based evaluation and find substantial variation in alignment with human judgments. Overall, MENTORQA establishes mentorship focused QA as a distinct research problem and provides a benchmark for studying agentic architectures and evaluation design in educational AI. The dataset and evaluation framework are released at https://aim-scu.github.io/MentorQA/.