A chain-of-thought (CoT) rationale can be coherent and answer-consistent while remaining causally unused. We call this phenomenon shortcut reasoning: the model appears to reason through CoT, but its answer does not depend on the expressed trace. We formalize shortcut reasoning as causal redundancy and propose a dual-disruption framework for measuring shortcut tendencies in CoT by testing two complementary aspects of causal explanation: necessity and sufficiency. We operationalize them as two behavioral signals: whether the answer changes under CoT disruption, and whether the answer can be recovered from the CoT after masking the original input. We instantiate this framework with CORD (Causal Redundancy of Reasoning via Disruption), an instance-level score that combines both signals. The two axes reduce complementary ambiguities in single-axis tests: answer preservation under CoT disruption can reflect shortcut use or redundant truthful reasoning, while early answer recovery can reflect shortcut use or a compact truthful rationale. Across 6 models and 2 datasets, CORD reveals a training-paradigm effect: R1-Distill models exhibit roughly twice the shortcut tendency of non-distilled models, while the two axes capture related but distinct signals. Together, these results motivate joint tests of necessity and sufficiency for detecting shortcut tendencies that can remain hidden behind correct answers.

Traditional text-based human-AI interactions typically follow a strict turn-taking approach. This rigid structure limits conversational flow, unlike natural human conversations, which can freely incorporate overlapping speech. However, our pilot study suggests that even in text-based interfaces, overlapping behaviors such as backchanneling and proactive responses lead to more natural and functional exchanges. Motivated by these findings, we introduce text-based overlapping interactions as a new challenge in human-AI communication, characterized by real-time typing, diverse response types, and interruptions. To enable AI systems to handle such interactions, we define three core tasks: deciding when to overlap, selecting the response type, and generating utterances. We construct a synthetic dataset for these tasks and train OverlapBot, an LLM-driven chatbot designed to engage in text-based overlapping interactions. Quantitative and qualitative evaluations show that OverlapBot increases turn exchanges compared to traditional turn-taking systems, with users making 72% more turns and the chatbot 130% more turns, which is perceived as efficient by end-users. This finding supports overlapping interactions and enhances communicative efficiency and engagement.

While machines learn from existing corpora, humans have the unique capability to establish and accept new language systems. This makes human form unique language systems within social groups. Aligning with this, we focus on a gap remaining in addressing translation challenges within social groups, where in-group members utilize unique terminologies. We propose KpopMT dataset, which aims to fill this gap by enabling precise terminology translation, choosing Kpop fandom as an initiative for social groups given its global popularity. Expert translators provide 1k English translations for Korean posts and comments, each annotated with specific terminology within social groups’ language systems. We evaluate existing translation systems including GPT models on KpopMT to identify their failure cases. Results show overall low scores, underscoring the challenges of reflecting group-specific terminologies and styles in translation. We make KpopMT publicly available.

The math word problem (MWP) is a complex task that requires natural language understanding and logical reasoning to extract key knowledge from natural language narratives. Previous studies have provided various MWP datasets but lack diversity in problem types, lexical usage patterns, languages, and annotations for intermediate solutions. To address these limitations, we introduce a new MWP dataset, named DMath (Diverse Math Word Problems), offering a wide range of diversity in problem types, lexical usage patterns, languages, and intermediate solutions. The problems are available in English and Korean and include an expression tree and Python code as intermediate solutions. Through extensive experiments, we demonstrate that the DMath dataset provides a new opportunity to evaluate the capability of large language models, i.e., GPT-4 only achieves about 75% accuracy on the DMath dataset.