KG-based fact verification verifies the truthfulness of claims by retrieving evidence graphs from the knowledge graph. The *faithful inference chains*, which are precise relation paths between the mentioned entities and evidence entities, retrieve precise evidence graphs addressing poor performance and weak logic for fact verification. Due to the diversity of relation paths, existing methods rarely extract faithful inference chains. To alleviate these issues, we propose Multi-view Heterogeneous Graph with Causal Intervention (MHGCI): (i) We construct a Multi-view Heterogeneous Graph enhancing relation path extraction from the view of different mentioned entities. (ii) We propose a self-optimizing causal intervention model to generate assistant entities mitigating the out-of-distribution problem caused by counterfactual relations. (iii) We propose a grounding method to extract evidence graphs from the KG by faithful inference chains. Experiments on the public KG-based fact verification dataset FactKG demonstrate that our model provides precise evidence graphs and achieves state-of-the-art performance.

Large language models (LLMs) have shown great capabilities in various tasks but also exhibited memorization of training data, raising tremendous privacy and copyright concerns. While prior works have studied memorization during pre-training, the exploration of memorization during fine-tuning is rather limited. Compared to pre-training, fine-tuning typically involves more sensitive data and diverse objectives, thus may bring distinct privacy risks and unique memorization behaviors. In this work, we conduct the first comprehensive analysis to explore language models’ (LMs) memorization during fine-tuning across tasks. Our studies with open-sourced and our own fine-tuned LMs across various tasks indicate that memorization presents a strong disparity among different fine-tuning tasks. We provide an intuitive explanation of this task disparity via sparse coding theory and unveil a strong correlation between memorization and attention score distribution.

Document-level Relation Extraction (DocRE) aims to extract relations among entity pairs in documents. Some works introduce logic constraints into DocRE, addressing the issues of opacity and weak logic in original DocRE models. However, they only focus on forward logic constraints and the rules mined in these works often suffer from pseudo rules with high standard-confidence but low support. In this paper, we proposes Bidirectional Constraints of Beta Rules(BCBR), a novel logic constraint framework. BCBR first introduces a new rule miner which model rules by beta contribtion. Then forward and reverse logic constraints are constructed based on beta rules. Finally, BCBR reconstruct rule consistency loss by bidirectional constraints to regulate the output of the DocRE model. Experiments show that BCBR outperforms original DocRE models in terms of relation extraction performance (~2.7 F1 score) and logical consistency(~3.1 logic score). Furthermore, BCBR consistently outperforms two other logic constraint frameworks.