Tomas Turek
Author directory2026
Can Multimodal Large Language Models Identify Thematic Maps in Scientific Articles?
Tomas Turek | Vincent Nguyen | Dhirendra Singh | Sarvnaz Karimi | Xiuzhen Zhang
Proceedings of the 3rd Workshop on Natural Language Processing Meets Climate Change (ClimateNLP 2026)
Tomas Turek | Vincent Nguyen | Dhirendra Singh | Sarvnaz Karimi | Xiuzhen Zhang
Proceedings of the 3rd Workshop on Natural Language Processing Meets Climate Change (ClimateNLP 2026)
Information seeking systems have yet to access multimodal information about thematic maps in scientific literature. Thematic maps are images of geographical maps showing themes of information and are crucial for understanding the impact of climate on natural resources. Although Multimodal Large Language Models (MLLMs) are applied in different applications, their effectiveness in identifying thematic maps is not explored. We address this gap by evaluating some of the state-of-the-art MLLMs to (1) classify thematic maps using different visual and textual contexts; (2) explain their predictions and uncover their sources of bias using attribute discovery; and (3) assist humans with the annotation of thematic maps in the scientific literature. We release a new public dataset (https://github.com/tomtx/cp-thematic-maps) and use it to show that MLLMs that were investigated in our study have room for improvement. We observe that MLLMs can be biased toward visual attributes of thematic maps (graphical indicators). The best classification result is an 80% strict F1 score by the Grok 4.3 model, which can fluctuate between 76% and 84% due to classification bias. Overall, we consistently find that the input of images with captions and examples of the correct answers yields substantial gains in task performance.
2025
CSIRO LT at SemEval-2025 Task 8: Answering Questions over Tabular Data using LLMs
Tomas Turek | Shakila Tonni | Vincent Nguyen | Huichen Yang | Sarvnaz Karimi
Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)
Tomas Turek | Shakila Tonni | Vincent Nguyen | Huichen Yang | Sarvnaz Karimi
Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)
Question Answering over large tables is challenging due to the difficulty of reasoning required in linking information from different parts of a table, such as heading and metadata to the values in the table and information needs. We investigate using Large Language Models (LLM) for tabular reasoning, where, given a pair of a table and a question from the DataBench benchmark, the models generate answers. We experiment with three techniques that enables symbolic reasoning through code execution: a direct code prompting (DCP) approach, ‘DCP_Py’, which uses Python, multi-step code (MSC) prompting ‘MSC_SQL+FS’ using SQL and ReAct prompting, ‘MSR_Py+FS’, which combines multi-step reasoning (MSR), few-shot (FS) learning and Python tools. We also conduct an analysis exploring the impact of answer types, data size, and multi-column dependencies on LLMs’ answer generation performance, including an assessment of the models’ limitations and the underlying challenges of tabular reasoning in LLMs.