As more than 70% of reviews in the existing opinion summary data set are positive, current opinion summarization approaches are hesitant to generate negative summaries given the input of negative texts. To address such sentiment bias, a direct approach without the reliance on a specific structure is to generate additional data based on large language models to balance the emotional distribution of the dataset. However, large-scale data augmentation based on large language models faces an apparent disadvantage, the expensive costs. Therefore, in this paper, we propose LASS, a novel data augmentation framework based on both LArge and Small language models for debiaSing opinion summarization. Specifically, a small number of synthesized negative reviews is obtained by rewriting the positive text via a large language model. Then, a disentangle reconstruction model is trained based on the generated data. After training, a large amount of synthetic data can be obtained by decoding the new representation obtained from the combination of different sample representations and filtering based on perplexity degree and sentiment classification. Experiments have proved that LASS can effectively alleviate emotional bias, similar to using only large models, but in a more economical way.

Cross-domain fake news detection, aiming to detect fake news in unseen domains, has achieved promising results with the help of pre-trained language models. Existing approaches mainly relied on extracting domain-independent representations or modeling domain discrepancies to achieve domain adaptation. However, we found that the relationship between entities in a piece of news and its corresponding label (fake or real) fluctuates among different domains. Such discrepancy is ignored by existing methods, leading to model entity bias. Therefore, in this paper, we propose a novel cross-domain fake news detection method based on dual-granularity adversarial training from the perspective of document-level and entity-level. Specifically, both the news pieces and their entities are modeled individually to construct an encoder that can generate domain-independent representations using adversarial training. Moreover, the dual-granularity soft prompt, consisting of two independent learnable segments trained on the source domains, is employed to make the model easily adapt to the unseen target domains. In addition, MultiFC, a released dataset for cross domain fake news detection, is not suitable for the evaluation due to its unreasonable domain construction rules. We artificially reconstructed the dataset and named it New-MultiFC, which is a more domain-discriminative dataset. Experimental results on both the newly constructed New-MultiFC and FND3 show the effectiveness of the proposed approach, achieving the state-of-the-art results in unseen domains.

As in the existing opinion summary data set, more than 70% are positive texts, the current opinion summarization approaches are reluctant to generate the negative opinion summary given the input of negative opinions. To address such sentiment bias, two approaches are proposed through two perspectives: model-specific and model-agnostic. For the model-specific approach, a variational autoencoder is proposed to disentangle the input representation into sentiment-relevant and sentiment-irrelevant components through adversarial loss. Therefore, the sentiment information in the input is kept and employed for the following decoding which avoids interference of content information with emotional signals. To further avoid relying on some specific opinion summarization frameworks, a model-agnostic approach based on counterfactual data augmentation is proposed. A dataset with a more balanced emotional polarity distribution is constructed using a large pre-trained language model based on some pairwise and mini-edited principles. Experimental results show that the sentiment consistency of the generated summaries is significantly improved using the proposed approaches, while their semantics quality is unaffected.

Approaches for unsupervised opinion summarization are generally based on the reconstruction model and generate a summary by decoding the aggregated representation of inputs. Recent work has shown that aggregating via simple average leads to vector degeneration, generating the generic summary. To tackle the challenge, some approaches select the inputs before aggregating. However, we argue that the selection is too coarse as not all information in each input is equally essential for the summary. For example, the content information such as “great coffee maker, easy to set up” is more valuable than the pattern such as “this is a great product”. Therefore, we propose a novel framework for unsupervised opinion summarization based on text representation disentanglement with counter-template. In specific, a disentangling module is added to the encoder-decoder architecture which decouples the input text representation into two parts: content and pattern. To capture the pattern information, a counter-template is utilized as supervision, which is automatically generated based on contrastive learning. Experimental results on two benchmark datasets show that the proposed approach outperforms the state-of-the-art baselines on both quality and stability.