Jingru Chen

Author directory

2026

The democratization of LLMs has accelerated the generation and circulation of highly fluent disinformation, making traditional syntax-semantic verification increasingly insufficient. Such deception often combines persuasive rhetoric, emotional manipulation, and narrative role construction to shape readers’ interpretations. However, existing detectors typically emphasize isolated signals—such as syntax, external knowledge, persuasion, or affective cues—and thus struggle to capture multi-faceted manipulation or provide structured, taxonomy-grounded evidence. To address this gap, we present ExTax, a taxonomy-aligned framework for interpretable disinformation detection. ExTax unifies persuasive rhetoric, emotional manipulation, and narrative roles into a 17-dimensional taxonomic space comprising 6 persuasive-rhetoric strategies, 5 emotional-manipulation methods, and 6 narrative-role categories. It elicits attributes from multiple frontier LLMs, reconciles their disagreements through Entropy-driven Dynamic Label Smoothing, and fuses the resulting taxonomic representations with contextual encodings via Heterogeneous Multi-Head Attention. Across five cross-domain and cross-genre benchmarks, ExTax achieves an overall Macro F1 of 0.8456, outperforming strong deep learning and LLM-based baselines. It also remains robust under severe genre imbalance, where the strongest deep baseline drops from 0.9454 to 0.6194 across genres. Our code is available at https://github.com/owlsan49/ExTax.
Large language model (LLM) evaluation is increasingly confounded by *variant contamination*: training corpora contain semantically equivalent but lexically or syntactically altered test items. These variants can evade detectors based on sampling consistency or perplexity, inflating benchmark scores through memorization rather than reasoning. We formalize this problem and introduce **DVD** (**D**etection via **V**ariance of generation **D**istribution), a detector requiring only the test item, without external reference corpora or paired contaminated/clean sets. DVD models the local output distribution through repeated temperature sampling. Our key insight is that contaminated items alternate between *memory-adherence* and *perturbation-drift* states, producing unusually high variance in the synthetic difficulty of low-probability tokens, whereas uncontaminated items remain in drift with comparatively stable difficulty values. We construct the first dedicated variant-contamination benchmark across Omni-MATH and SuperGPQA by generating and filtering semantically equivalent variants, and simulate contamination by fine-tuning Qwen2.5 and Llama3.1 models of different scales. In most evaluated settings across datasets, model scales, architectures, and training configurations, **DVD** outperforms perplexity-based, Min-*k*%++, edit-distance (CDD), and embedding-similarity baselines, while showing strong robustness to hyperparameters. These results support generation-distribution variance as a principled and practical signal for detecting variant contamination, helping improve the reliability of LLM evaluation.

2024

Humor is a fundamental facet of human cognition and interaction. Yet, despite recent advances in natural language processing, humor detection remains a challenging task that is complicated by the scarcity of datasets that pair humorous texts with similar non-humorous counterparts. We investigate whether large language models (LLMs) can generate synthetic data for humor detection via editing texts. We benchmark LLMs on an existing human dataset and show that current LLMs display an impressive ability to “unfun” jokes, as judged by humans and as measured on the downstream task of humor detection. We extend our approach to a code-mixed English-Hindi humor dataset where we find that GPT-4’s synthetic data is highly rated by bilingual annotators and provides challenging adversarial examples for humor classifiers.