Agentic autonomy is essential for large language models (LLMs) to function as agentic AI systems, yet existing benchmarks often emphasize operational aspects, or HOW an agent behaves, such as tool use. The ability to reconsider the user’s underlying purpose, or WHY the agent acts, remains underexplored. We define this capability as "purpose-reflective autonomy" and introduce CaseInterviewBench, an interactive, interview-style benchmark inspired by consultant recruitment. The benchmark contains 104 predefined scenarios, each with public background context, an initial request, hidden facts, and ground-truth problem formulations and resolutions. The target LLM asks questions to uncover hidden facts, then identifies the actual problem and proposes a resolution, enabling us to observe how it infers, revises, and refines its understanding of the scenario’s underlying purpose in a structured setting. We evaluate hidden-fact disclosure, final problem formulation, and resolution using rule-based metrics and LLM-as-a-judge assessments. Across several frontier and open-weight models, we found substantial room for improvement. We also observed severe violations in which some models fabricated facilitator responses during the benchmark flow, potentially creating serious risks in agentic execution.

Repository-level repair requires not only correct patches but also inspectable records that explain how issue requirements are translated into code changes and post-edit evidence. We contribute THEMIS, a stage-aware repair workflow that externalizes this requirement-to-repair process through semantic interpretation, a runtime requirement–code graph, graph-derived Developer guidance, retained repair rationale and patches, and post-edit audit records. A retrospective audit of 300 SWE-bench Lite cases demonstrates that these artifacts provide broad support for cross-stage inspection: a complete Developer rationale is available for 288 cases, and 214 cases (71.3%) retain a complete audited field set connecting the selected stages. The retained records further enable systematic measurement of cross-stage correspondence: target symbols recur in 62.6% of Developer rationales and in 62.8% of patches, rising to 75.8% when related symbols are included. In a paired 100-case comparison, the relational workflow resolves 19 cases versus 9 for the direct same-input condition, corresponding to a 10-percentage-point improvement. Together, these results show that THEMIS turns otherwise implicit requirement-to-repair transitions into observable traces, enabling systematic inspection of how repair decisions persist, align, and evolve across stages.

The human-level performance of Large Language Models (LLMs) across various tasks has raised expectations for the potential of artificial intelligence (AI) to possess emotions someday. To explore the capability of current LLMs to express emotions in their outputs, we conducted an experiment using several LLMs (OpenAI GPT, Google Gemini, Meta Llama3, and Cohere Command R+) to role-play as agents answering questions with specified emotional states. We defined the emotional states using Russell’s Circumplex model, a well-established framework that characterizes emotions along the sleepy-activated (arousal) and pleasure-displeasure (valence) axes. We chose this model for its simplicity, utilizing two continuous parameters, which allows for better controllability in applications involving continuous changes in emotional states. The responses generated were evaluated using a sentiment analysis model, independent of the LLMs, trained on the GoEmotions dataset. The evaluation showed that the emotional states of the generated answers were consistent with the specifications, demonstrating the LLMs’ capability for emotional expression. This indicates the potential for LLM-based

We present a tendency of large language models (LLMs) to generate absurd patterns despite their clear inappropriateness in a simple task of identifying regularities in number series. Several approaches have been proposed to apply LLMs to complex real-world tasks, such as providing knowledge through retrieval-augmented generation and executing multi-step tasks using AI agent frameworks. However, these approaches rely on the logical consistency and self-coherence of LLMs, making it crucial to evaluate these aspects and consider potential countermeasures. To identify cases where LLMs fail to maintain logical consistency, we conducted an experiment in which LLMs were asked to explain the patterns in various integer sequences, ranging from arithmetic sequences to randomly generated integer series. While the models successfully identified correct patterns in arithmetic and geometric sequences, they frequently over-recognized patterns that were inconsistent with the given numbers when analyzing randomly generated series. This issue was observed even in multi-step reasoning models, including OpenAI o3, o4-mini, and Google Gemini 2.5 Flash Preview Thinking. This tendency to perceive non-existent patterns can be interpreted as the AI model equivalent of Idola Tribus and highlights potential limitations in their capability for applied tasks requiring logical reasoning, even when employing chain-of-thought reasoning mechanisms.