Text Style Transfer: Leveraging a Style Classifier on Entangled Latent Representations

Learning a good latent representation is essential for text style transfer, which generates a new sentence by changing the attributes of a given sentence while preserving its content. Most previous works adopt disentangled latent representation learning to realize style transfer. We propose a novel text style transfer algorithm with entangled latent representation, and introduce a style classifier that can regulate the latent structure and transfer style. Moreover, our algorithm for style transfer applies to both single-attribute and multi-attribute transfer. Extensive experimental results show that our method generally outperforms state-of-the-art approaches.


Introduction
Text generation, which leverages knowledge in computational linguistics and artificial intelligence for automatically generating natural language texts, is the core problem for a number of Natural Language Processing (NLP) applications such as speech to text, conversational/dialogue system (Banchs and Li, 2012;Kim et al., 2007), and text summarization (Ozsoy et al., 2011;Liu et al., 2018). Text style transfer can be thought of as a controllable text generation task, which aims to restyle a given sentence by changing specific attributes (sentiment, tense, formality, or politeness) while preserving the remaining attributes and the content. Successful applications of text style transfer include paraphrasing (Han et al., 2017), formality transfer (Rao and Tetreault, 2018), and text simplification (Cao et al., 2020).
A good latent representation is essential to the performance of text style transfer. Regarding the structure of the latent representation, the current work for text style transfer can be generally categorized into the disentangled representation and the entangled representation. In particular, the former method aims to learn disentangled latent representations by separating the style information from the content, while the latter method learns latent representations that entangle the style with the content. Disentangled representations are often interpretable and consequently most of the current work adopts this method (Hu et al., 2017;Zhao et al., 2018;John et al., 2019;Bao et al., 2019). However, learning disentangled representations is often challenging; and multiple attributespecific decoders are commonly required for text generation, which is undesirable especially when transferring multiple attributes. The entangled representations, on the other hand, has been shown to achieve promising performance on the content preservation and to produce fluent sentences with a much less complicated architecture (Lample et al., 2019;Wang et al., 2019;Liu et al., 2020).
Although existing models achieve adequate performance on text style transfer, most of them are designed specifically for style transfer (Hu et al., 2017;Shen et al., 2017;Lample et al., 2019;John et al., 2019;Bao et al., 2019;Wang et al., 2019), and meanwhile lack of explicit modeling of the latent space. We argue that the quality of latent representations is crucial for text generation. In this study, we focus on building a generative model that supports both text style transfer and text generation with regularized entangled latent representations.
Our contributions can be summarized as follows: (1) We extend the framework of adversarial autoencoder by including a classifier for both the regularization of the latent space and text style transfer. We show that the classifier can divide sentences with different attributes into different regions in the latent space and thus greatly improve the performance of style transfer. (2) We provide algorithms for both single-attribute and multi-attribute style transfer. We empirically compare with sev-eral state-of-the-art baselines and show that our proposed method achieves promising results.

Related Work
In this section, we first introduce several Generative Adversarial Network (GAN)-regularized autoencoders, which have been successfully used for text manipulation. Then we review some recent methods for text style transfer focusing on the latent representation.

Probabilistic Generative Autoencoders
GANs (Goodfellow et al., 2014) are popular generative models consisting of two basic components: a generator for generating new samples and a discriminator for distinguishing real samples from generated samples. Makhzani et al. (2015) introduce adversarial autoencoders (AAEs), which turn basic autoencoders into probabilistic models. The encoder E φ maps the input x to a latent representation z, z = E φ (x). The decoder D θ reconstructs the input from z asx = D θ (z). The discriminator D w is introduced to distinguish between z and samples from a prior distribution P z . The objective of AAEs is formulated below: Shen et al. (2020) adopt AAEs and introduce denoising adversarial autoencoders (DAAEs) with a smoother structure of latent space. Based on the Wasserstein autoencoders (WAEs) (Tolstikhin et al., 2018;Zhao et al., 2018) propose adversarially regularized autoencoders (ARAE) by extending AAEs for discrete sequences. Unlike AAEs which use a fixed prior distribution, Zhao et al. (2018) adopt a learnable prior parameterized by the generator of a GAN . Particularly, the discriminator and the generator are first learned by using the latent representation from the encoder. The discriminator is then used to adversarially train the encoder by minimizing the discrepancy between the posterior and the prior.

Methods for Text Style Transfer
Disentangled latent representation Most work on text style transfer is based on learning disentangled latent representations (Hu et al., 2017;Shen et al., 2017;John et al., 2019;Bao et al., 2019), where the attributes are separated from the content. For example, to generate a sentence with desired attributes, the decoder in (Hu et al., 2017) takes the style-independent latent representation and the desired style as the input. Shen et al. (2017) adopts adversarial training by using a binary CNN-based discriminator to determine whether a generated sentence is successfully transferred or not.  use a target domain language model instead of a conventional binary classifier as the discriminator.
Entangled latent representation On the contrary, some recent work proposes to learn latent representations that entangle the style with the content. Although the learning of disentangled latent representations is unnecessary, other mechanisms are needed to guide the style transfer. For example, Lample et al. (2019) apply the back-translation mechanism (referred to as BTDAE) and the algorithm achieves the state-of-the-art performance on the content preservation. Wang et al. (2019) transfer text style by updating the latent representation (referred to as TAE) based on the Fast Gradient Sign Method (FGSM) . Liu et al. (2020) also use a gradient-based optimization to update the latent representation.
Style classifier A classifier is commonly used in a style discriminator to enforce the desired style (Hu et al., 2017;Tian et al., 2018). For example, ARAE (Zhao et al., 2018) imposes a style classifier on the latent representation of a sentence to ensure the transferred sentence containing the target attribute.
The main differences between our method and ARAE (Zhao et al., 2018) can be summarized as follows: 1) Representation structure: the latent representation of ARAE can be considered as disentangled while ours is entangled. 2) Style classifier: the style classifier we adopt helps the clustering of latent representations based on attributes while the classifier in ARAE only enforces the target contribute on transferred sentences. 3) To realize style transfer, ARAE uses the classifier to train the encoder net adversarially, such that the latent representation of the given text could contain the information of the target attribute. However, in our method, the style transfer is realized by directly modifying the latent representation. 4) Due to the adversarial training process ARAE requires multiple decoders to perform style transfer, while our method can use only one decoder.

Our Method
We first briefly explain text style transfer as follows. Generally, a source or an input sentence includes both the content and the attribute. In Figure 1, we use (x, y) to include both the input x and the attribute y. As a concrete example, a sentence "this place is a great place to live !" has two types of attribute: the positive sentiment and the present tense. The positive sentiment is reflected by the word "great" and the present tense is reflected by the word "is". All remaining words in this sentence are considered as the content. For single-attribute style transfer, only one attribute (e.g. sentiment) will be transferred, and the other attributes and the content will be kept the same. In the above example with the sentiment style transfer, the sentence will be converted into a negative sentence: "this place is a terrible place to live !" by flipping the sentiment label y from positive to negative. In contrast, for multi-attribute style transfer, two or more attributes will be transferred simultaneously while the rest will be preserved. Again, in the above example the sentence will be converted into "this place was a terrible place to live !" by transferring both the sentiment and the tense.

Network Architecture
We propose a generative model that can be used for both text style transfer and text generation. The network architecture of our method is illustrated in Figure 1, where the top one is for training and the bottom one is for style transfer. The network for training includes three parts: an autoencoder, a GAN, and a style classifier. Specifically, the autoencoder is learned to reconstruct the input sentences. The discriminator of the GAN is to distinguish the aggregated posterior of the encoder from the prior, which is modeled by the generator of the GAN. The style classifier uses the latent representation as the input, and classifies latent representations based on their attributes. The classifier can also be used in style transfer, which will be explained later.

Objective Function for Training
The overall objective function for training includes three parts: the reconstruction loss, the adversarial loss induced by the GAN, and the classification loss. Similar to ARAE (Zhao et al., 2018), we use the Wasserstein distance to measure the discrep- ancy between two distributions. Denote the parameters of the encoder, the decoder, the discriminator, the generator, and the classifier as φ, θ, w, ψ, and φ c , respectively. The overall objective function is defined as follows.
In the above expressions, the variablez is the output of the generator G ψ (s), where the noise s ∈ N (0, I); y is the label of the source attribute; and the critic function f w of the discriminator is obtained by a min-max optimization: We summarize the training algorithm in Algorithm 1. First, the autoencoder is trained by minimizing the reconstruction loss, i.e., min φ,θ L rec (φ, θ).
Next, based on the latent representation from the encoder, the encoder and the style classifier are jointly trained by minimizing the classification loss, i.e., min φ,φc L clas (φ, φ c ). Meanwhile, the critic function f w and the generator of the GAN are learned via the min-max optimization min ψ max w L crit (ψ, w). Finally, the critic function f w is utilized to adversarially train the encoder, i.e., min φ L crit (φ). We emphasize that we do not explicitly disentangle the attributes from the content in the latent representation. Therefore, to implement style transfer, the style classifier is crucial, which guides the clustering of the entangled latent representations based on their attributes.
Inputs: Px input distribution; E φ encoder; G ψ generator; fw discriminator/critic function for each training iteration do // Train the encoder and decoder for reconstruction(φ, θ).

Style Transfer
After training the network, we can implement text style transfer (as shown at the bottom of Figure 1). We summarize the algorithm for style transfer in Algorithm 2, which works for both single-attribute and multi-attribute style transfer. Normally for multi-attribute style transfer, multiple style classifiers are required: each corresponding to an attribute. To make the network scalable, we instead use a single style classifier by combining the labels of attributes. In this case, each attribute label corresponds to an attribute-combination (such Algorithm 2: Transfer Algorithm.
well-trained classifier C φc ; the initial weights w = {wj}; decay coefficient λ; target attribute y ; threshold t; maximal iterations I; attribute vector v; the weight of attribute vector k.
as present-positive, past-positive, present-negative, and past-negative with both the tense and the sentiment as the attributes for transferring).
To perform style transfer, given an input sentence, we first get its latent representation as the output of the encoder. With the entangled latent representation, the key of style transfer is how to update the latent representation of the source sentence. To achieve that, we adopt two different but commonly used updates: the fast gradient based and the vector arithmetic based. To obtain the target sentence with the desired attribute, we then feed the updated latent representation to the decoder.
Fast gradient based: FGSM is employed by Wang et al. (2019) to update the latent representation for style transfer. Concretely, the latent representation is updated along the gradient of the classification loss with the step size w. A set w contains a few step sizes with an increasing order. We sequentially test these step sizes until obtaining the desired latent representation. This is to maximally preserve the content of the sentence and also to prevent the modification of the latent presentation from falling into a local optimum. In each iteration, the updated latent representationẑ f is given as follows: where L clas represents a style classifier loss, C φc is a well-trained classifier, and y represents the target label. The detailed algorithm is displayed in Method 1 of Algorithm 2.
Vector arithmetic based: In several studies e.g., Zhao et al. (2018); Shen et al. (2020), the latent vector arithmetic based method is employed in text style transfer or text interpolation. Specifically, the latent representation z of the source sentence is modified by an attribute vector "v". For example, assume that the source attribute is positive. When transferring the attribute from positive to negative we can update z by z − v; and when transferring from negative to positive we can update z by z + v. The same as Shen et al. (2020), the attribute vector "v" uses the mean of the latent representations of 100 samples with the source attribute and 100 samples with the target attribute from the validation set. For multi-attribute transfer, the attribute vector v is computed in the same way. The only difference is that the label of the source attribute and the target attribute corresponds to an attribute-combination as explained before. The updated latent representation z v can be formulated as follows: where k is a hyperparameter denoting the weight associated with the attribute vector.

Experiments
In this section, we first visualize the latent representation of our method, and then compare our method with several baselines, namely, TAE (Wang et al., 2019), ARAE (Zhao et al., 2018), and DAAE (Shen et al., 2020) for single-attribute and multipleattribute text style transfer. We then evaluate our model on text generation and compare it with ARAE.

Datasets
We use Yelp and Amazon datasets for evaluation.
Yelp: This dataset consists of Yelp restaurant and business reviews , which includes 444K training samples, 4K validation samples, and 1K test samples.
Amazon: This dataset includes product reviews from Amazon (He and McAuley, 2016), which includes 555K training samples, 2K validation samples, and 1K test samples.
On both Yelp and Amazon datasets, reviews with a rating score above three are considered as positive samples, otherwise are considered as negative samples.

Experimental setups
In our experiment, similar to ARAE (Zhao et al., 2018), we use one layer LSTM with 200 hidden units for both the encoder and the decoder. Both the generator and the discriminator in the GAN use simple MLP networks. The style classifier is built by a shallow MLP network with two hidden layers, as our experiment indicates that too many layers can degrade the performance of the classifier.
The weighting parameters λ w and λ c are set to 0.1 on Yelp and 1 on Amazon. In the fast gradient method, the set of the initial weights w is set to {0.005, 0.006, 0.007, 0.008, 0.009, 0.01}, where the weights are ordered increasingly.

Evaluation
Following previous studies, for both automatic and human evaluations, we assess the performance of style transfer from three perspectives: transfer control, content preservation, and fluency. In automatic evaluation, three commonly used metrics are adopted: the transfer rate, the BLEU score, and the Perplexity (PPL) score.
Transfer control: It evaluates whether the style of the source sentences is correctly flipped. The transfer rate is the percentage of the corrected transferred sentences, and we use a fastText classifier (Joulin et al., 2017) to determine that.
Content preservation: It evaluates how the content is preserved in the transferred sentences. We use n-gram statistics (4-gram) of the BLEU score (Papineni et al., 2002) to quantify the content preservation against the references .
Fluency: It evaluates the grammatical structure and the naturalness of the generated (or transferred) text sentences. We use a language model KenLM (Heafield, 2011) to calculate the PPL score of text sentences for evaluating fluency.

Visualization
We show the projected latent representation of both the source and the target sentences and compare it with TAE (Wang et al., 2019). To better show the structure of the latent representation, we use the visualization tool t-SNE (van der Maaten and Hinton, 2008) in 2-dimension. Figure 2(a) shows the latent representations of 1000 source samples with positive and negative labels. Figure 2 In our framework, both the GAN and the style classifier help the regularization of the latent representation. Figure 2(a) indicates that in our method, the latent representations tend to form two clusters. Specifically, the positive samples tend to locate at the bottom while the negative samples tend to locate on the top. In contrast, the positive and the negative samples in TAE are generally mixed together. In Figure 2(c), as the value of the step size w increases in our method, the latent representation of the positive samples tends to move towards the bottom, which corresponds to the position of the positive cluster. In contrast, the latent representation of the negative samples tends to move towards the top, which corresponds to the position of the negative cluster. This observation clearly shows the guidance of the style classifier on clustering the latent representations. In comparison, in Figure  2(b), without the GAN and the style classifier in TAE, the latent representation of each target sample needs to be updated along different directions.

Evaluation of Latent Representation via K-nearest-neighbours
It is desirable that close latent representations lead to semantically similar sentences after feeding latent representations to the decoder. Such property indicates the smoothness of the latent space. In this experiment, we find k = 9 nearest neighbours of the latent representation of a sentence "service is terrible and won't return.", and then generate sentences by feeding these latent representations to the decoder. It is expected that the generated sentences are close to the source sentence in terms of the sentiment attribute and the content. For comparison, we consider four differ network architectures and  show the generated sentences in Table 1. • TAE: The generated sentences contain both the positive and the negative samples. Moreover, many of them have different contents that are related to "place" or "location" instead of the source content "service" or "return".
• TAE+GAN: We regularize the latent representation in TAE by a GAN. Although all sentences are related to the source content "service" their sentiment attributes are largely different.
• TAE + classifier: We add a style classifier in TAE. Different from TAE+GAN, the generated sentences have the same negative attribute but some sentences deviate from the source content "service" or "return".
• Our method: Our network architecture includes both the GAN and a style classifier.
All generated sentences have the same negative attribute and are related to the source content "service" or "return". Table 1: Evaluation of the smoothness of the latent space via k-nearest-neighbours. The source sentence is "service is terrible and won't return ".

TAE TAE + GAN
service was terrible . service is great and friendly . their service is terrible .
service isn't that great either . service is great and friendly .
service is mediocre and slow . this place is terrible ! service is slow and horrible . service is slow and horrible .
service is lacking and food is mediocre . this location is terrible .
the service is friendly and fast . service is always good .
prices are good and the service is great . service was bad .
service is quick and friendly . everything was great and i will return ! the service is always friendly and good .
TAE + classifier guidance Our method service was terrible . service is n't that great either . their service is terrible .
i wo n't be back . food was terrible .
the service is n't frequent enough . this place is terrible .
will not return to this place . service is slow and horrible .
needless to say we wo n't be back ! the waiter was terrible .
service was n't too bad -nice people . i wo n't be back .
the service was not that professional ! service is mediocre and slow .
the service did n't get any better . terrible service .
service is n't too bad , but could be better .

Single-Attribute Style Transfer
We compare our method with TAE by using FGSM to update the latent representation since TAE is only designed with FGSM (Table 2). TAE has a very low transfer rate in experiments. This however leads to high BLEU scores and low PPL scores as most target sentences are the same as the source sentences. By contrast, our method can successfully transfer most sentences, and leads to decent BLEU and PPL scores. We also compare the performance of our methods with ARAE and DAAE using the vector arithmetic based update on latent representations for style transfer. As mentioned before, the vector arithmetic based update can be used to evaluate the smoothness of the latent space. In Table 3, we compare with two baselines on both Yelp and Amazon and display the results that achieve the best trade-off among the three evaluation metrics. The hyperparameter k of vector arithmetic method is chosen based on the performance in the validation set. Our method achieves the highest transfer rate and a comparable BLEU score with ARAE on Yelp, while ARAE achieves the lowest PPL score. On Amazon, our method obtains the best performance on the transfer rate and the BLEU score with a slightly higher PPL score than ARAE. By contrast, DAAE does not perform well on both datasets especially on Amazon. FGSM and vector arithmetic method for style transfer have their pros and cons. Table 4 shows the evaluation results of both FGSM based and vector arithmetic based methods on Yelp data. Generally, for both methods, as the step size w or v increases, the transfer rate is improving, while the performance of BLEU and PPL are decreasing. In the case of the FGSM based method, the best trade-off is when w is set as 0.007, while the vector arithmetic based method has the best trade-off when v is set as 1.5. With w = 0.007 and v = 1.5, FGSM based method achieves better transfer rate and BLEU score but the lower performance of PPL than vector arithmetic based method. From our experiment, we also observe that FGSM based style transfer needs much longer updating time in testing than the vector arithmetic based method. Although the automatic evaluation metrics e.g., the BLEU score, are widely used, they sometimes do not well align with the human judgement (Ma et al., 2018). Therefore, to fully evaluate the performance we also carried out the human evaluation on sentiment transfer and compare it with ARAE on Yelp. We use the vector arithmetic based update of the latent representation on the first 200 positive and 200 negative sentences in the test set. Three annotators were recruited and provided scores in the range of 1∼5 regarding the transfer control, the content preservation and the fluency. The Kappa statistic of the agreement between raters in the human evaluation is 0.657. The average scores over three annotators are shown in Table 5, and our method generally outperforms ARAE.

Multi-Attribute Style Transfer
We also evaluate our model for multi-attribute transfer and compare with ARAE on Yelp. The goal of multi-attribute style transfer is to transform multiple attributes in a sentence at once while preserving the main content of the sentence. Using the same example sentence "this place is a great place to live !" with positive sentiment and present tense, multi-attribute transfer converts it into a sentence with the negative sentiment and the past tense "this place was a terrible place to live !". In the training phase, the style classifier uses both the latent vector of a given sentence and the original attribute label as the inputs; while in style transfer, the style classifier uses both the latent vector and the desired attribute label as the inputs. In pre-processing of single-attribute style transfer, the attribute is labelled as either "0" or "1". In multi-attribute style transfer (e.g. tense and sentiment), each attribute combination will be defined as an individual class (e.g. present-positive: "0", past-positive: "1", present-negative: "2", and pastnegative: "3").
In particular, we use the Stanford Parser to extract the main verb of a sentence from Yelp and then determine the tense of a sentence based on its part-of-speech tag (POS tags) (Klein and Manning, 2003). Table 6 shows the evaluation results of style transfer for two attributes: sentiment and tense transfer. In our model, we test on two network variants, one consisting of two style classifiers each corresponding to one attribute, and the other consists of only one style classifier that combines both attributes into one label as described before. The results show that our method is superior to ARAE even with one style classifier. When using two classifiers to realize multiple attributes transfer, for example, tense and sentiment transfer, each classifier is responsible for transferring one attribute. Specifically, after the sentiment-classifier transferred sentiment attribute, the transferred sentences will be the inputs of the tense-classifier for transferring tense attribute. As using two classifiers, each classifier only needs to transfer one attribute, having less pressure of transferring two attributes together, it achieves higher accuracy than one classifier case. However, since it requires two steps transfer, the self BLEU score decreases slightly.

Evaluation of Text Generation
Unlike most of the current models for style transfer, our model can also be used to generate new text sentences owing to the introduction of the latent prior distribution. To generate a new sentence, we first take the noise s as the input to the generator of the GAN and get a latent representation. Then we feed the latent representation to the decoder and obtain the new sentence.
In previous work, both LSTM (Zhao et al., 2018;Lample et al., 2019;Shen et al., 2020) and transformer (Wang et al., 2019) have been used as the base network in the encoder and decoder architecture. Hence, we further evaluate the performance of our method based on these two networks on Yelp and show the results in Table 7. Experimental results indicate that both the transformer-based networks and the LSTM-based networks in our method achieve a similar trade-off among the three evaluation metrics. The transformer-based method achieves higher BLEU scores but lower transfer rates and higher PPL scores, while the LSTMbased method leads to a better performance on the transfer rate and the PPL score, but a worse performance on the BLEU score. We evaluate the quality of the text generation using the LSTM-based network in our method by comparing 10, 000 generated sentences with sentences generated by ARAE on Yelp. The experimental results in Table 8 show that our method achieves a better degree of fluency. In particular, the PPL score of our method is lower than that of ARAE by around 10% (the PPL score of our method and ARAE is 76.83 and 86.98, respectively). Table 8: Generated sentences of our method and ARAE.
Our method the woman who could give up the store says you are very picky . the wait staff is great but overall i did n't like the customer. i will not recommend this place to any women in future . the man was always great and the service was really helpful . do not waste a star from the older man this place is overpriced . the store experience is awesome the salesman it was very nice. oh ok and the man in the service looked nice . kind of really nice man 's walking the restaurant that they 're very delish .
ARAE the gentleman i left inside the kitchen was a rather nice follow up . this woman has gotten me . the woman in a little job of perfect ! the man was not that amazing if i tried to order it . this woman has a cake must me in the burgh . all was excellent by the salesman we had to do . there is a friendly man and the crowd of bacon in your face . their woman was under staffed as very polite and how talented .
Text samples of ARAE are from Zhao et al. (2018).

Style Classifier in Other Models
Through the above experiments, we have illustrated the effect of the style classifier in our method on clustering the latent representations based on the attributes. We also perform an ablation study regarding the style classifier on two other advanced models: DAAE and BTDAE (Lample et al., 2019).
In DAAE, we implement style transfer by the vector arithmetic based update, and in BTDAE the back-translation algorithm is used for style transfer. From Table 9, we observe that with the inclusion of a style classifier in DAAE the performance on all evaluation metrics is improved. For BTDAE, with a style classifier the BLEU and the PPL scores are improved. These results again demonstrate the effectiveness of a style classifier on style transfer. As the official code of BTDAE is unavailable, we implemented the algorithm based on the description in Lample et al. (2019).

Conclusion and Future Work
In this paper, we proposed a new approach for text style transfer with entangled latent representations. We added a classifier to regularize the distribution of latent sentences in a probabilistic autoencoder. Extensive experiments show that this regularized latent structure significantly improves the downstream text manipulation tasks. Compared with benchmarks our method achieves impressive results on both single-attribute and multi-attribute text style transfer. Moreover, both approaches of fast gradient and vector arithmetic style transfer outperform baselines on style transfer tasks. In addition, we demonstrated that the classifier regularization also improves other style transfer models.
In the future, we would like to explore other methods to regularize latent representation in controllable text generation. Moreover, text generation models have a wide range of applications in NLP tasks. Besides style transfer, we will apply our model to other tasks such as text simplification and examine the latent structure in these applications.