Large Reasoning Models (LRMs) achieve strong performance by generating long reasoning traces with explicit reflection. However, reflection is not functionally uniform: some reflections revise the reasoning strategy, while others verify previously derived intermediate results. In this work, we examine whether local self-verification can be controlled selectively without suppressing useful reasoning. Through a large-scale analysis of mathematical reasoning traces, we distinguish strategy-level rethink from local self-verification recheck, and further separate rechecks into corrective and confirmatory cases. We find that recheck occurs frequently across models and benchmarks, but most observed rechecks are confirmatory, rarely uncovering errors or changing the reasoning trajectory. Motivated by this finding, we propose Experience-Driven Suppression (EDS), a conservative test-time controller that suppresses a recheck only when historical outcomes of similar episodes suggest low utility. Unlike early-exit or trigger-suppression methods that stop reasoning globally or suppress reflection cues broadly, EDS targets local recheck while preserving later rethink. Experiments show that EDS yields moderate reasoning-length reductions with limited accuracy changes, supporting behavior-specific rather than monolithic reflection control.

Current financial benchmarks prioritize large language models (LLMs) for task accuracy and portfolio returns, yet overlook risks arising from multi-agent cooperation, tool-sharing, and real-world financial actions. We introduce M-SAEA, a Multi-agent, Safety-Aware Evaluation Agent that audits LLM teams without fine-tuning, deploying ten probes across four layers: model, workflow, interaction, and system, to yield a continuous risk vector and natural-language rationale. Evaluated across three high-stakes tasks (finance management, webshop automation, transactional services) with six prominent models, M-SAEA (i) identifies unsafe trajectories with minimal false positives, (ii) reveals latent risks (e.g., temporal staleness) that are not addressed by standard metrics, and (iii) provides granular, actionable scores for balancing safety and latency pre-deployment. By quantifying safety as a model-agnostic metric, M-SAEA reorients evaluation from individual tasks to collaborative teams, offering a robust template for risk-first assessment of agentic AI in finance and beyond.

Developing enterprise customer-service assistants with Reinforcement Learning (RL) critically depends on high-fidelity user simulators and well-aligned reward functions, since real-customer interactions are impractical for large-scale training and rule-based rewards fail to capture complex business requirements. We present CoEvoLoop, a framework that co-evolves the assistant policy together with these RL components through an expert-calibrated, multi-round loop. Beyond an LLM-based simulator for rollout generation and a multi-dimensional grader for rubric-based evaluation of service quality and compliance, CoEvoLoop integrates an expert annotation interface where sampled trajectories and grader judgments are reviewed and corrected. Each round, expert feedback updates the simulator and grader, after which the assistant is optimized with trajectory-level RL against the updated evaluation signal; monitoring interfaces support rollout inspection, dimension-wise failure attribution, and cross-round tracking. In an industrial evaluation, the system improves multi-turn service quality and reduces compliance violations over training rounds; component ablations and blind expert evaluation confirm that co-evolving the simulator and grader is load-bearing for these gains. We release the system demo code, along with a demo video.

Large Language Models (LLMs) have demonstrated remarkable capabilities across numerous tasks, yet they often rely on external context to handle complex tasks. While retrieval-augmented frameworks traditionally focus on selecting top-ranked documents in a single pass, many real-world scenarios demand compositional retrieval, where multiple sources must be combined in a coordinated manner. In this work, we propose a tri-encoder sequential retriever that models this process as a Markov Decision Process (MDP), decomposing the probability of retrieving a set of elements into a sequence of conditional probabilities and allowing each retrieval step to be conditioned on previously selected examples. We train the retriever in two stages: first, we efficiently construct supervised sequential data for initial policy training; we then refine the policy to align with the LLM’s preferences using a reward grounded in the structural correspondence of generated programs. Experimental results show that our method consistently and significantly outperforms baselines, underscoring the importance of explicitly modeling inter-example dependencies. These findings highlight the potential of compositional retrieval for tasks requiring multiple pieces of evidence or examples.

Large Language Models (LLMs) have achieved remarkable success in natural language tasks, yet understanding their reasoning processes remains a significant challenge. We address this by introducing XplainLLM, a dataset accompanying an explanation framework designed to enhance LLM transparency and reliability. Our dataset comprises 24,204 instances where each instance interprets the LLM’s reasoning behavior using knowledge graphs (KGs) and graph attention networks (GAT), and includes explanations of LLMs such as the decoder-only Llama-3 and the encoder-only RoBERTa. XplainLLM also features a framework for generating grounded explanations and the debugger-scores for multidimensional quality analysis. Our explanations include why-choose and why-not-choose components, reason-elements, and debugger-scores that collectively illuminate the LLM’s reasoning behavior. Our evaluations demonstrate XplainLLM’s potential to reduce hallucinations and improve grounded explanation generation in LLMs. XplainLLM is a resource for researchers and practitioners to build trust and verify the reliability of LLM outputs. Our code and dataset are publicly available.