Yingzhe Fan
Author directory2026
Overview of the LT4CPR 2026 Shared Task: Generating Situation Reports from Social Media Messages
Fei Xia | Haotian Zhu | Yun Woo Lee | Eitan Grenev | Allison K Burton | Yingzhe Fan | William D. Lewis | Antonios Anastasopoulos | Belu Ticona
Proceedings of the First Workshop on Language Technologies for Crisis Preparedness and Response
Fei Xia | Haotian Zhu | Yun Woo Lee | Eitan Grenev | Allison K Burton | Yingzhe Fan | William D. Lewis | Antonios Anastasopoulos | Belu Ticona
Proceedings of the First Workshop on Language Technologies for Crisis Preparedness and Response
Language Technologies for Crisis Preparedness and Response (LT4CPR) encompass a broad range of natural language applications designed to support disaster response and preparedness. Within this domain, our shared task focuses specifically on leveraging social media data to extract actionable information and generate structured situation reports. To establish a formal benchmark, we introduce a domain-specific taxonomy for categorizing critical crisis information, alongside a novel pipeline for generating synthetic datasets to support task training and evaluation. Furthermore, we develop two baseline systems and propose an evaluation methodology designed to leverage the structural framework of situation reports. The shared task attracted participation from nine teams, whose diverse approaches offer valuable preliminary insights into the feasibility of structured situation report generation and the effectiveness of our evaluation setup. Finally, we highlight key open challenges (such as data access, operational alignment, noise filtering) which needs to be addressed in future research.
Stable Scores, Failed Retrieval: Diagnosing RAG Evaluation in Low-Resource Indic Languages
Priyanshi Garg | Jasmin Gustafson | Yingzhe Fan
Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026)
Priyanshi Garg | Jasmin Gustafson | Yingzhe Fan
Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026)
Automatic metrics stand in for native-speaker evaluation in low-resource retrieval-augmented generation (RAG), so their validity determines what can be concluded about these systems. We test whether common metrics detect re- trieval and grounding failure in one fixed RAG pipeline across ten Indic languages. They do not merely miss the failure in Santali; they invert it. Hit@5 is 0.011, yet raw XLM-R BERTScore is the highest in the set at 0.902, changes little when retrieved evidence is re- moved, and assigns nearly the same score to matched and unrelated references. The cause is coverage: the scorer maps 39% of Santali tokens to <unk>, and a scorer whose vocab- ulary includes Ol Chiki restores discrimina- tion while LaBSE does not. Retrieval fails for the same reason. The gold article is indexed for 80% of Santali queries, but the retriever shares XLM-R’s tokenizer and collapses before reranking. Judge-based faithfulness and con- text precision, computed by a model with a dif- ferent tokenizer, flag the failure independently. A metric that cannot represent a script does not become noisy; it becomes constant and high. We argue that metric validity in low-resource languages must be established per language, and that evidence removal, reference shuffling, and the scorer’s <unk> rate can be important additions.