Ziao Wang
2025
Test-Time Steering for Lossless Text Compression via Weighted Product of Experts
Qihang Zhang
|
Muchen Li
|
Ziao Wang
|
Renjie Liao
|
Lele Wang
Findings of the Association for Computational Linguistics: EMNLP 2025
Lossless compression techniques are crucial in an era of rapidly growing data. Traditional universal compressors like gzip offer low computational overhead, high speed, and broad applicability across data distributions. However, they often lead to worse compression rates than modern neural compressors, which leverage large-scale training data to model data distributions more effectively.Despite their advantages, neural compressors struggle to generalize to unseen data. To address this limitation, we propose a novel framework that performs Test-Time Steering via a Weighted Product of Experts (wPoE).At inference, our method adaptively combines a universal compression model with a pretrained neural language model, ensuring the compression rate is at least as good as the best individual model.Extensive experiments demonstrate that our approach improves the performance of text compression without requiring fine-tuning. Furthermore, it seamlessly integrates with any autoregressive language model, providing a practical solution for enhancing text compression across diverse data distributions.
2021
Building the Directed Semantic Graph for Coherent Long Text Generation
Ziao Wang
|
Xiaofeng Zhang
|
Hongwei Du
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Generating long text conditionally depending on the short input text has recently attracted more and more research efforts. Most existing approaches focus more on introducing extra knowledge to supplement the short input text, but ignore the coherence issue of the generated texts. To address aforementioned research issue, this paper proposes a novel two-stage approach to generate coherent long text. Particularly, we first build a document-level path for each output text with each sentence embedding as its node, and a revised self-organising map (SOM) is proposed to cluster similar nodes of a family of document-level paths to construct the directed semantic graph. Then, three subgraph alignment methods are proposed to extract the maximum matching paths or subgraphs. These directed subgraphs are considered to well preserve extra but relevant content to the short input text, and then they are decoded by the employed pre-trained model to generate coherent long text. Extensive experiments have been performed on three real-world datasets, and the promising results demonstrate that the proposed approach is superior to the state-of-the-art approaches w.r.t. a number of evaluation criteria.
Search
Fix author
Co-authors
- Hongwei Du 1
 - Muchen Li 1
 - Renjie Liao 1
 - Lele Wang 1
 - Xiaofeng Zhang 1
 - show all...