2026

Large language models are increasingly regarded as strong translators, yet it remains unclear what their success establishes beyond overall translation quality. This question is especially consequential in Chinese web novel translation, where sentence-level cultural meaning shapes tone, characterization, and reader experience. Existing evaluation provides only indirect evidence, as benchmarks emphasize overall quality and available metrics do not explicitly assess the cultural and pragmatic phenomena underlying reader experience. We introduce DITING (a mythological creature known for discerning truth), the first comprehensive evaluation framework for this problem, supported by an expert-annotated benchmark of over 18K Chinese-English sentence pairs, a reasoning-driven multi-agent evaluator, AgentEval, and a 1,440-pair meta-evaluation benchmark, MetricAlign. AgentEval achieves the strongest correlation with human judgments among eight representative automatic metrics. Applying DITING to 14 translation systems shows that current progress is selective rather than general: frontier LLMs lead overall, but their gains concentrate in a subset of dimensions, while zero-pronoun resolution remains the clearest bottleneck. Overall, our results show that aggregate rankings substantially overstate how broadly sentence-level cultural meaning has been solved in web novel translation. All the code and data in https://github.com/whunextgen/DITING.
Model calibration aligns a model’s confidence with the accuracy of its predictions, directly influencing reliability and user trust. Surprisingly, existing research on knowledge distillation has paid little attention to the calibration of student models. In this paper, we evaluate calibration across diverse knowledge distillation settings and demonstrate that student models distilled from black-box teachers generally exhibit poor calibration. We attribute this to the fact that distillation fails to convey well-calibrated confidence, which serves as a crucial expression of predictive uncertainty. To address this limitation, we propose an ***U**ncertainty-**I**ntegrated **K**nowledge **D**istillation* (**UIKD**) method comprising two key components: (1) *uncertainty estimation*, which leverages the collective intelligence of multiple LLMs via a debate mechanism to quantify uncertainty in the distillation data; and (2) *uncertainty integration*, which employs an adaptive weighted loss function to integrate this data uncertainty into the student model effectively. Experiments on common-sense reasoning and natural language inference benchmarks demonstrate that UIKD consistently improves student model calibration while maintaining predictive accuracy. Our code is available at https://github.com/RuixuanZhang127/UIKD.

2023

Knowledge graph entity typing (KGET) aims at inferring plausible types of entities in knowledge graphs. Existing approaches to KGET focus on how to better encode the knowledge provided by the neighbors and types of an entity into its representation. However, they ignore the semantic knowledge provided by the way in which types can be clustered together. In this paper, we propose a novel method called Multi-view Contrastive Learning for knowledge graph Entity Typing MCLET, which effectively encodes the coarse-grained knowledge provided by clusters into entity and type embeddings. MCLET is composed of three modules: i) Multi-view Generation and Encoder module, which encodes structured information from entity-type, entity-cluster and cluster-type views; ii) Cross-view Contrastive Learning module, which encourages different views to collaboratively improve view-specific representations of entities and types; iii) Entity Typing Prediction module, which integrates multi-head attention and a Mixture-of-Experts strategy to infer missing entity types. Extensive experiments show the strong performance of MCLET compared to the state-of-the-art

2022

We investigate the knowledge graph entity typing task which aims at inferring plausible entity types. In this paper, we propose a novel Transformer-based Entity Typing (TET) approach, effectively encoding the content of neighbours of an entity by means of a transformer mechanism. More precisely, TET is composed of three different mechanisms: a local transformer allowing to infer missing entity types by independently encoding the information provided by each of its neighbours; a global transformer aggregating the information of all neighbours of an entity into a single long sequence to reason about more complex entity types; and a context transformer integrating neighbours content in a differentiated way through information exchange between neighbour pairs, while preserving the graph structure. Furthermore, TET uses information about class membership of types to semantically strengthen the representation of an entity. Experiments on two real-world datasets demonstrate the superior performance of TET compared to the state-of-the-art.

2021

2020

Recently, a variety of neural models have been proposed for lyrics generation. However, most previous work completes the generation process in a single pass with little human intervention. We believe that lyrics creation is a creative process with human intelligence centered. AI should play a role as an assistant in the lyrics creation process, where human interactions are crucial for high-quality creation. This paper demonstrates Youling, an AI-assisted lyrics creation system, designed to collaborate with music creators. In the lyrics generation process, Youling supports traditional one pass full-text generation mode as well as an interactive generation mode, which allows users to select the satisfactory sentences from generated candidates conditioned on preceding context. The system also provides a revision module which enables users to revise undesired sentences or words of lyrics repeatedly. Besides, Youling allows users to use multifaceted attributes to control the content and format of generated lyrics. The demo video of the system is available at https://youtu.be/DFeNpHk0pm4.