Ruoyu Xiang

Author directory

2026

Financial language understanding requires accurate interpretation of numerical disclosures, implicit corporate intent, and evidence distributed across languages and documents. We present FinEval 2026, a shared task at the FinNLP workshop at EMNLP 2026, comprising Greek financial named-entity recognition, Japanese implicit communication recognition, and multilingual financial question answering. The registration records contain 35 teams after reconciliation, and 14 teams contribute system description papers. The final leaderboards contain 7, 10, and 15 entries for the three subtasks, respectively. We describe the task design, participation, submitted approaches, and final results. Leading scores reach 0.9676 entity F1, 0.9714 accuracy, and 0.3827 ROUGE-1 F1. Systems combine domain rules, language-specific modeling, structured prompting, retrieval, and controlled answer revision. Their results highlight the value of task-specific constraints while exposing limitations arising from small test sets, development-to-test shifts, and reuse of previously released reference material. We identify priorities for more reliable multilingual financial evaluation.
Real-world financial analysis involves information across multiple languages and modalities, from reports and news to scanned filings and meeting recordings. Yet most existing evaluations of LLMs in finance remain text-only, monolingual, and largely saturated by current models. To bridge these gaps, we present MultiFinBen, the first expert-annotated multilingual (five languages) and multimodal (text, vision, audio) benchmark for evaluating LLMs in realistic financial contexts. MultiFinBen introduces two new task families: multilingual financial reasoning, which tests cross-lingual evidence integration from filings and news, and financial OCR, which extracts structured text from scanned documents containing tables and charts. Rather than aggregating all available datasets, we apply a structured, difficulty-aware selection based on advanced model performance, ensuring balanced challenge and removing redundant tasks. Evaluating 21 leading LLMs shows that even frontier multimodal models like GPT-4o achieve only 46.01% overall, stronger on vision and audio but dropping sharply in multilingual settings. These findings expose persistent limitations in multilingual, multimodal, and expert-level financial reasoning. All datasets, evaluation scripts, and leaderboards are publicly released.

2025

Despite Greece’s pivotal role in the global economy, large language models (LLMs) remain underexplored for Greek financial context due to the linguistic complexity of Greek and the scarcity of domain-specific datasets. While multilingual financial NLP has revealed large performance gaps across languages, no benchmarks or LLMs have been tailored for Greek financial tasks until now. To bridge this gap, we introduce Plutus-ben, the first Greek Financial Evaluation Benchmark, and Plutus-8B, the first financial LLM fine-tuned on Greek-specific financial data. Plutus-ben addresses six core tasks: numeric/textual named entity recognition, question answering, extractive summarization, abstractive summarization, and topic classification. To support these tasks, we release four new expert-annotated Greek financial datasets and incorporate two existing resources. Our comprehensive evaluation of 24 LLMs reveals persistent challenges in Greek financial NLP, driven by linguistic complexity, domain terminology, and financial reasoning gaps. Experiment results underscore the limitations of cross-lingual transfer and the need for Greek-specific financial modeling. We publicly release Plutus-ben, Plutus-8B, and all associated datasets to promote reproducible research and advance multilingual financial NLP.

2024