Trying to disentangle a complicated feature space into a simpler latent representation
Â
Code available at this repo
Â
Beta-VAE (higgins et al. 2017) - adds hyperparameter​ beta to weight the compactness prior term
Â
Beta-VAE H (burgess et al. 2018) - adds hyperparameter C ​ to control the compactness prior term
Â
Â
Â
Factor-VAEÂ (kim & minh, 2018) - adds total correlation loss term
Â
Beta-Total-Correlation VAE (chen et al. 2018) - same objective as factor-vae, but computed without a discriminator
Â
TRIMÂ (singh et al. 2020) - yields attribution on transformations to learn simpler representations
Â
Â
Â
Â
(chen et al. 2016)
(khemakhem et al. 2020)
encourages accurate reconstruction of the input (note could do this w/ something smarter than pixel loss) |
encourages points to be compactly placed in space;
this term can be further divided into 3 terms:
encourages latent variables to be independent
encourages mutual info between input and latent code to be high for a subset of the latent variables |
solving
maximize non-gaussianity of z or minimize mutual info between its components
preserves information between the latent space + input |
encourages latent space to be decoupled |
details + code
VAEÂ
(kingma & welling, 2013)
ALAE
(pidhorskyi et al. 2020)
StyleGan + StyleGan2
(karras et al. 2019)
disentangles by using latent representation at different scales |
+ TRIM loss
(singh et al. 2020)
penalizes interpretations to be desirable (e.g. sparse, monotonic)
Â
+ prediction loss
(singh et al. 2020)
if we are given a trained predictor, we can minimize its error rather than simply reconstructing the input