Large language models (LLMs) are increasingly required to process inputs composed of multiple context units, such as answer options, retrieved passages, or in-context demonstrations. Effective reasoning in such scenarios demands capturing both intrinsic semantics within each unit and the aggregated semantics across units. However, recent efforts typically treat multiple units as a single entangled sequence, ignoring the distinct knowledge within isolated units, which leads to suboptimal performance. To this end, we propose MUA(Multiple Context Units Attention), a dual-view self-attention module that computes a local view for learning unit-specific semantics and a global view to enable bidirectional interaction across units. These two views share projection matrices and are fused by a lightweight input-conditioned gate, while generation proceeds with standard causal decoding. With an optimized implementation, this design achieves efficient fusion of multiple context units and superior information utilization during input encoding at marginal overhead. Experiments on diverse multiple context units reasoning benchmarks, including multiple-choice, multi-hop QA and language understanding tasks, show that MUA consistently improves performance across model scales, exhibits increased robustness to context units reordering, and establishes new state-of-the-art results on key benchmarks.

Large Language Models (LLMs) have significantly improved the performance of unsupervised Event Argument Extraction (EAE) tasks. However, LLMs’ inherent preferences severely hinder their effectiveness in EAE, leading to what we term preference traps, namely, the Prior Knowledge Trap, the Sycophancy Hallucination Trap, and the Output Contradiction Trap. Existing approaches often fall into these traps due to misalignments between their prior knowledge, instructions, or output constraints and LLMs’ preferences, which significantly limits further performance gains. To address this issue, we propose Choose-After-Think (CAT), an unsupervised EAE framework designed to handle these preference traps through targeted measures. CAT innovatively divides the EAE task into two phases: identification of event information (argument roles) (Think Phase) and selection of the final answers from a candidate set (Choose Phase). This two-phase approach reduces the impact of individual token probability anomalies and ensures the integrity of EAE results. Experimental results demonstrate that CAT (based on the local 7B model, zero-shot setting) matches the performance of the best DeepSeek-R1 API model, with a significantly lower time cost.