Industrial standard documents combine hierarchical section structures, table-based rules, and complex conditional statements; as a result, the same expression can refer to different rules depending on where it appears in a document. However, existing text-based approaches and conventional knowledge graph (KG) construction methods do not jointly preserve these structures, making them brittle on complex rule-based queries. In this work, we propose an ontology-based KG construction method that formalizes section structure as an ontology and integrates it with rule-aware triplets extracted from text and tables. To evaluate our approach, we construct IndusSpec-QA, a QA dataset based on three industrial standard documents, and compare the proposed method against a range of baselines. Experimental results show that the proposed Ontology-based KG-RAG outperforms existing methods, with particularly large gains on table-based rule questions and multi-condition reasoning. These results demonstrate the importance of knowledge representations that jointly preserve section and rule structures for QA over industrial standard documents. The code is publicly available at: https://github.com/Ontology-KG/ontology_based_kg_paper

Large language models have demonstrated strong performance across a wide range of tasks, yet they remain prone to generating incorrect information with high confidence. To mitigate this reliability issue, it is necessary to properly assess model confidence before answer generation. However, existing methods either rely on post-hoc estimation based on generated tokens or sampling results, or require additional training based on supervised learning, which limits their computational efficiency and generalizability. In this paper, we propose PReemptive Internal Confidence Estimator (PRICE), which estimates confidence before answer generation by leveraging internal signals of LLMs that can be extracted from a single forward pass. PRICE combines signals that capture pattern agreement across attention heads and the semantic clarity of hidden states. Through experiments on diverse models and benchmarks, we show that PRICE achieves competitive performance against classifier-based methods without requiring additional training. These results suggest that PRICE can contribute to improving both the efficiency of LLMs and their safety in high-risk domains.

Time-Series Question Answering (TSQA) aims to interpret time-series information and generate answers according to the intent of natural language questions. However, in TSQA, the required temporal granularity and reasoning scope vary across queries, making it crucial to effectively organize the time-series information needed for each query. In this study, we propose TiMER (**Ti**me-series and language alignment with **M**ulti-granularity **E**xpert **R**outing), a time-series-language alignment framework that selectively constructs multi-granularity time-series information according to query intent and connects it to the LLM input space. Experimental results on a large-scale TSQA benchmark show that TiMER achieves an overall accuracy of 81.02% using only a small number of external alignment modules while keeping the LLM frozen, outperforming various LLM-based baselines. In addition, extensive ablation studies verify the effectiveness of each component. This study presents an effective direction for natural language-based time-series understanding and temporal reasoning by dynamically organizing and utilizing time-series information according to query intent. The code is publicly available at https://github.com/hyuna0926/TiMER_paper.

ESG analysis requires complex language understanding across diverse sources, including corporate reports, news, and disclosures, for tasks such as ESG relevance detection, action interpretation, impact analysis, and question answering. However, existing ESG datasets are fragmented by task and source, with heterogeneous formats and annotation schemes, limiting their use for training general-purpose ESG LLMs and unified evaluation. In this work, we construct ESG-Instruct, a 30,000-instance ESG instruction dataset built from public ESG resources, and ESGU-Bench, an ESG understanding benchmark covering six representative ESG tasks. Based on these resources, we further develop ESGPT, an 8B ESG-specialized LLM, and evaluate it against diverse open-source and API-based LLMs. Experimental results show that general-purpose LLMs exhibit substantial performance variation across ESG tasks, while ESGPT, despite its relatively small scale, achieves the best performance on multiple ESG tasks. This work provides unified data and benchmark resources for training and evaluating ESG-specialized LLMs.

Large Language Model (LLM) personalization is challenging because user preferences are rarely stated explicitly; instead, they emerge as implicit, multifaceted signals in users’ past interaction histories. Existing methods often compress user histories into a single representation or retrieve and inject past records without explicitly linking them to supporting evidence, which can lead to hallucinated or overly generic outputs. In this work, we propose Multi-Aspect PersonaLized Modeling with Evidence (MAPLE). MAPLE structures implicit user preferences through a MULTI-ASPECT PREFERENCE PROFILE and connects them to target-specific grounding evidence through a PERSONALIZED EVIDENCE PLAN. By selecting only generation points that are both relevant to the user and supported by available evidence, MAPLE improves personalization and grounding faithfulness simultaneously. Experiments on the Games and Books domains with two backbone LLMs show that MAPLE consistently outperforms strong baselines and generalizes robustly under limited-history and unseen-user settings.

Humans attempt to understand the real world by asking the fundamental question ”Why?” when faced with incomprehensible situations in everyday life. Such why-questions provide essential knowledge that can help in understanding these situations. In this study, we conducted an end-to-end process to verify the utility of consecutive why-questions, from constructing a large language model (LLM)-based dataset to performing quantitative evaluation and analysis. Firstly, we created a WHY-Chain dataset, consisting of answers generated by an LLM in response to chain-of-why-questions, including a validity check. We also incorporated objectives that effectively capture the ”consecutive” characteristic of the data. Using the WHY-Chain dataset and two types of self-supervised objectives, we trained the pre-trained model. As a result, the refined model demonstrated improved performance on downstream tasks that require commonsense reasoning. Additionally, we conducted various ablation studies to assess the impact of different factors, confirming the scalability of the proposed approach. Lastly, we confirmed the consistency of the logical information by reasoning chain analysis of the answers generated from consecutive why-questions.

Question Answering (QA) with large language models has shown impressive performance, yet hallucinations still persist, particularly when user queries carry incorrect premises, insufficient context, or linguistic ambiguity. To address this issue, we propose Lightweight Query Checkpoint (LQC), a small classification model that detects verification-required queries before the LLM generates a potentially faulty answer. LQC leverages hidden states extracted from intermediate layers of a smaller-scale, non-instruct-tuned LLM to effectively distinguish queries requiring verification from clear queries. We first systematically define categories of queries that need verification, construct a dataset comprising both defective and clear queries, and train a binary contrastive learning model. Through extensive experiments on various QA datasets, we demonstrate that incorporating LQC into QA pipelines reduces hallucinations while preserving strong answer quality.

Consistency within a document is a crucial feature indicative of its quality. Recently, within the vast amount of information produced across various media, there exists a significant number of low-quality documents that either lack internal consistency or contain content utterly unrelated to their headlines. Such low-quality documents induce fatigue in readers and undermine the credibility of the media source that provided them. Consequently, research to automatically detect these low-quality documents based on natural language processing is imperative. In this study, we introduce a hierarchical graph convolutional network (HGCN) that can detect internal inconsistencies within a document and incongruences between the title and body. Moreover, we constructed the Inconsistency Dataset, leveraging published news data and its meta-data, to train our model to detect document inconsistencies. Experimental results demonstrated that the HGCN achieved superior performance with an accuracy of 91.20% on our constructed Inconsistency Dataset, outperforming other comparative models. Additionally, on the publicly available incongruent-related dataset, the proposed methodology demonstrated a performance of 92.00%, validating its general applicability. Finally, an ablation study further confirmed the significant impact of meta-data utilization on performance enhancement. We anticipate that our model can be universally applied to detect and filter low-quality documents in the real world.

With the expansion of pre-trained language model usage in recent years, the importance of datasets for performing tasks in specialized domains has significantly increased. Therefore, we have built a Korean dataset called ESG-Kor to automatically extract Environmental, Social, and Governance (ESG) information, which has recently gained importance. ESG-Kor is a dataset consisting of a total of 118,946 sentences that extracted information on each ESG component from Korean companies’ sustainability reports and manually labeled it according to objective rules provided by ESG evaluation agencies. To verify the effectiveness and applicability of the ESG-Kor dataset, classification performance was confirmed using several Korean pre-trained language models, and significant performance was obtained. Additionally, by extending the ESG classification model to documents of small and medium enterprises and extracting information based on ESG key issues and in-depth analysis, we demonstrated potential and practical use cases in the ESG field.

The news subheading summarizes an article’s contents in several sentences to support the headline limited to solely conveying the main contents. So, it is necessary to generate compelling news subheadings in consideration of the structural characteristics of the news. In this paper, we propose a subheading generation model using topical headline information. We introduce a discriminative learning method that utilizes the prediction result of masked headline tokens. Experiments show that the proposed model is effective and outperforms the comparative models on three news datasets written in two languages. We also show that our model performs robustly on a small dataset and various masking ratios. Qualitative analysis and human evaluations also show that the overall quality of generated subheadings improved over the comparative models.

The primary purpose of dialogue state tracking(DST), a critical component of an end-toend conversational system, is to build a model that responds well to real-world situations. Although we often change our minds from time to time during ordinary conversations, current benchmark datasets do not adequately reflect such occurrences and instead consist of over-simplified conversations, in which no one changes their mind during a conversation. As the main question inspiring the present study, “Are current benchmark datasets sufficiently diverse to handle casual conversations in which one changes their mind after a certain topic is over?” We found that the answer is “No” because DST models cannot refer to previous user preferences when template-based turnback utterances are injected into the dataset. Even in the the simplest mind-changing (turnback) scenario, the performance of DST models significantly degenerated. However, we found that this performance degeneration can be recovered when the turnback scenarios are explicitly designed in the training set, implying that the problem is not with the DST models but rather with the construction of the benchmark dataset.

Dialogue state tracking (DST) aims to extract essential information from multi-turn dialog situations and take appropriate actions. A belief state, one of the core pieces of information, refers to the subject and its specific content, and appears in the form of domain-slot-value. The trained model predicts “accumulated” belief states in every turn, and joint goal accuracy and slot accuracy are mainly used to evaluate the prediction; however, we specify that the current evaluation metrics have a critical limitation when evaluating belief states accumulated as the dialogue proceeds, especially in the most used MultiWOZ dataset. Additionally, we propose relative slot accuracy to complement existing metrics. Relative slot accuracy does not depend on the number of predefined slots, and allows intuitive evaluation by assigning relative scores according to the turn of each dialog. This study also encourages not solely the reporting of joint goal accuracy, but also various complementary metrics in DST tasks for the sake of a realistic evaluation.