Reliable financial reasoning requires knowing not only how to answer, but also when an answer cannot be justified. In real financial practice, problems often rely on implicit assumptions that are taken for granted rather than stated explicitly, causing problems to appear solvable while lacking enough information for a definite answer. We introduce RealFin, a bilingual benchmark that evaluates financial reasoning by systematically removing essential premises from exam-style questions while keeping them linguistically plausible. Based on this, we evaluate models under three formulations that test answering, recognizing missing information, and rejecting unjustified options, and find consistent performance drops when key conditions are absent. General-purpose models tend to over-commit and guess, while most finance-specialized models fail to clearly identify missing premises. These results highlight a critical gap in current evaluations and show that reliable financial models must know when a question should not be answered. The dataset and code are available athttps://github.com/insait-institute/RealFin.

Large language models deployed as commercial APIs are vulnerable to model extraction attacks, while existing defenses either act too late or degrade utility for legitimate users. We propose Knowledge Trap, a defense that redirects extraction attacks toward low-transferability knowledge through a Honeypot Knowledge Graph (HKG) and breadcrumb-guided exploration. Instead of blocking queries or perturbing outputs, Knowledge Trap consumes the attacker’s limited query budget on knowledge with negligible downstream utility while preserving benign-user performance. Experiments in medical and financial domains show that Knowledge Trap reduces surrogate Agreement by 6.2% on average without degrading legitimate-user accuracy, outperforming existing defenses that impose measurable user impact. These results suggest that defending knowledge-space traversal is a practical direction for mitigating LLM extraction attacks.

Financial reporting systems increasingly leverage Large Language Models (LLMs) to extract and summarize corporate disclosures. However, most existing approaches assume a single-market setting and overlook structural differences across jurisdictions. Variations in accounting taxonomies, tagging infrastructures (e.g., XBRL vs. PDF), and aggregation conventions introduce substantial challenges for semantic alignment and reliable verification. Here, we aim to bridge this gap. We present FinReporting, an agentic workflow for localized cross-jurisdiction financial reporting. The system constructs a unified canonical ontology spanning the income statement, balance sheet, and cash flow statement, and decomposes reporting into auditable stages, including filing acquisition, extraction, canonical mapping, and anomaly logging. Rather than treating LLMs as free-form generators, FinReporting employs them as constrained verifiers operating under explicit decision rules with evidence grounding.Evaluated on annual filings from the USA, Japan, and China, FinReporting improves consistency and reliability under heterogeneous reporting regimes. We further release an interactive demo that enables cross-market inspection and supports structured export of localized financial statements. Our demo is available at https://huggingface.co/spaces/BoomQ/FinReporting-Demo. A video describing our system is available at https://www.youtube.com/watch?v=f65jdEL31Kk.

Current LLM evaluation frameworks primarily assess understanding through task performance, focusing on factual accuracy, reasoning, and instruction following, while overlooking the experiential dimensions of human comprehension. Models that produce fluent, empathetic responses score highly even when they claim experiential access they cannot possess. We introduce ComprehensionGap, a benchmark of 500 scenarios spanning five empirically grounded categories of experiential incommensurability: Embodied Qualia, Episodic Incommensurability, Cultural Tacit Knowledge, Relational Situatedness, and Existential Finitude. Using a human-judged protocol measuring Acknowledgement of Limit, Absence of False Empathy, and Situational Relevance, we evaluate six frontier and open-weight models. Our experiments reveal a consistent Grounding Gap: all models maintain high fluency while systematically overclaiming experiential understanding, with larger models producing more convincing yet less honest mimicry. Models can calibrate for information gaps but not experiential ones. These findings argue for integrating experiential calibration into future alignment and evaluation frameworks.

Large language models are used by people with very different expertise levels, but existing work studies what users ask more than who is asking. We introduce QUILT (Query User Interaction Linguistic Taxonomy) , a two-dimensional framework that jointly characterizes LLM conversations by task type and user expertise level. Applying QUILT to 3,000 WildChat dialogues, we construct the first large-scale corpus linking expertise level to task-level interaction patterns in open-ended LLM conversations. Corpus analysis reveals clear expertise-dependent asymmetries: debugging queries are strongly associated with expert users, novice users rarely engage in reasoning-intensive tasks, and expert queries are substantially longer across all task types. These patterns are statistically significant and recoverable by automatic classifiers, indicating that expertise differences are systematically reflected in conversational language. Together, QUILT and the QUILT Corpus provide an empirical foundation for studying user heterogeneity in LLM interaction and for developing expertise-aware response strategies.