One of the main motivations for training high quality
image generative models is their potential use as tools for
image manipulation. Recently, generative adversarial networks (GANs)
have been able to generate images of remarkable quality.
Unfortunately, adversarially-trained unconditional generator networks
have not been successful as image priors.
One of the main requirements for a network to act as a generative image prior,
is being able to generate every possible image from the target distribution.
Adversarial learning often experiences mode-collapse, which manifests
in generators that cannot generate some modes of the target distribution.
Another requirement often not satisfied is invertibility i.e. having an efficient way of finding a valid
input latent code given a required output image.
In this work, we show that differently from earlier GANs,
the very recently proposed style-generators [18] are quite easy
to invert. We use this important observation to propose
style generators as general purpose image priors. We show
that style generators outperform other GANs as well as
Deep Image Prior as priors for image enhancement tasks.
The latent space spanned by style-generators satisfies linear identity-pose relations.
The latent space linearity, combined with invertibility, allows us to animate still facial images without supervision.
Extensive experiments are performed to support the main contributions of this paper.
arXiv 2019
Results
Inpainting
Corrupted
PGGAN [17]
Mescheder et al. [22]
DIP [28]
Ours
GT
Super-Resolution (128x128 to 1024x1024)
Bicubic
PGGAN [17]
Mescheder et al. [22]
DIP [28]
Ours
GT
Re-animation: Animating Obama from a video of Trump
BibTeX
@article{gabbay2019styleimageprior,
author = {Aviv Gabbay and Yedid Hoshen},
title = {Style Generator Inversion for Image Enhancement and Animation},
journal = {arXiv preprint arXiv:1906.11880},
year = {2019}
}
References
[17] T. Karras, T. Aila, S. Laine, and J. Lehtinen. Progressive growing of gans for improved quality, stability, and variation. In ICLR, 2018.
[18] T. Karras, S. Laine, and T. Aila. A style-based generator architecture for generative adversarial networks. In CVPR, 2019.
[22] L. Mescheder, S. Nowozin, and A. Geiger. Which training methods for gans do actually converge? In ICML, 2018.
[28] D. Ulyanov, A. Vedaldi, and V. Lempitsky. Deep image prior. In CVPR, 2018.
