wae-rnf-lm/ 000755 001750 001750 00000000000 13454442220 017335 5 ustar 00kingofspace0wzz kingofspace0wzz 000000 000000 wae-rnf-lm/README.md 000644 001750 001750 00000005153 13451622421 020620 0 ustar 00kingofspace0wzz kingofspace0wzz 000000 000000 # Riemannian Normalizing Flow on WAE Code for our NAACL2019 Paper "Riemannian Normalizing Flow on Variational Wasserstein Autoencoder for Text Modeling" https://arxiv.org/abs/1904.02399 Author: [Prince Zizhuang Wang](https://github.com/kingofspace0wzz/) and [William Yang Wang](http://www.cs.ucsb.edu/~william/)
An example when latent space does not reflect input space. Left: a manifold that is highly curved in the central region. The yellow line is the geodesic (shortest) path connecting two sample points shown on the manifold. Right: The projection of manifold into 2D latent space, where the color brightness indicates curvature with respect to the manifold. The green line is the geodesic path if taking the curvature into account, while the blue line is the geodesic path if we regard latent space as Euclidean. Middle: The corresponding geodesic paths projected back from latent space to manifold. The white line corresponds to the straight geodesic path in Euclidean space. It is far longer than the true geodesic on manifold since it does not take the curvature into account in latent space. ## Running the code ### Requirements * ``python 3.6`` * ``pytorch 1.0.0`` * ``spacy 2.0.12`` * ``torchtext 0.3.0`` ### Training train on ptb ```bash $ python main.py --kla --center --flow --mmd --enc_type lstm --de_type lstm --data data/ptb ``` train on yahoo ```bash $ python main.py --kla --center --flow --mmd --enc_type lstm --de_type lstm --data data/ptb ``` train on yelp ```bash $ python main.py --kla --center --flow --mmd --enc_type lstm --de_type lstm --data data/ptb ``` ### Options | Option | Usage | Value (Range) | | ------------- |:-------------| :-----| | kla | use kl annealing | True or False | | center | use clusters to compute MMD | True or False | | flow | use Normalizing Flow | True or False | | MMD | use Wasserstein distance | True or False| | enc_type | encoder model | lstm or gru | | de_type | decoder model | lstm or gru | ### Acknowledgement * [Sentence VAE, Bowman](https://arxiv.org/abs/1511.06349) * [von-Mises Fisher VAE, Xu](https://arxiv.org/abs/1808.10805) * [Dirichlet VAE for Text Modeling, Xiao](https://arxiv.org/abs/1811.00135) * [Dilated Convolutional VAE, Yang](https://arxiv.org/abs/1702.08139) wae-rnf-lm/clustering/ 000755 001750 001750 00000000000 13451246653 021525 5 ustar 00kingofspace0wzz kingofspace0wzz 000000 000000 wae-rnf-lm/clustering/pairwise.py 000644 001750 001750 00000001331 13452744422 023716 0 ustar 00kingofspace0wzz kingofspace0wzz 000000 000000 import torch def pairwise_distance(data1, data2=None, device=-1): if data2 is None: data2 = data1 if device!=-1: data1, data2 = data1.to(device), data2.to(device) #N*1*M A = data1.unsqueeze(dim=1) #1*N*M B = data2.unsqueeze(dim=0) dis = (A-B)**2.0 #return N*N matrix for pairwise distance dis = dis.sum(dim=-1).squeeze() return dis def group_pairwise(X, groups, device=0, fun=lambda r,c: pairwise_distance(r, c).cpu()): group_dict = {} for group_index_r, group_r in enumerate(groups): for group_index_c, group_c in enumerate(groups): R, C = X[group_r], X[group_c] if device!=-1: R = R.to(device) C = C.to(device) group_dict[(group_index_r, group_index_c)] = fun(R, C) return group_dict wae-rnf-lm/clustering/__pycache__/ 000755 001750 001750 00000000000 13451246654 023736 5 ustar 00kingofspace0wzz kingofspace0wzz 000000 000000 wae-rnf-lm/clustering/__pycache__/pairwise.cpython-36.pyc 000644 001750 001750 00000001661 13451102264 030177 0 ustar 00kingofspace0wzz kingofspace0wzz 000000 000000 3 eg\ @ s( d dl Z d ddZd dd fddZdS ) N c C sb |d kr| }|dkr*| j ||j | } }| jdd}|jdd}|| d }|jddj }|S )Nr )dimr g @r )cuda unsqueezesumsqueeze)Zdata1Zdata2deviceABdis r =/home/kingofspace0wzz/projects/wae-rnf/clustering/pairwise.pypairwise_distance s r c C s t | |j S )N)r cpu)rcr r r