The-GANfather

Make a model you can't refuse

Ian Goodfellow: Generative Adversarial Networks (NIPS 2016 tutorial)

Why study generative models?

Excellent to test of ability to use High-dimensional data
Simulated RL
Missing data
Realistic generation tasks
- Single Image Super-Resolution (Ledig et al 2010)
Adobe has an interactive GAN - iGAN where a user can draw a line, which would be translated to something - mountain by the generative model.
Image to Image Translation - edges to photos

What is a Markov Chain? What are Autoencoders?

There are two types for Explicit Density -> Approximate density

Variational
- Variational Autoencoder
Markov Chain
- Boltzmann Machine

One example of a fully visible belief net is Wavenet. Amazing quality but it takes two minutes to synthesize one second of audio. (Lot of time)

GANs

Use latent code
Asymptotically consistent
No Markov chains needed
Often regarded as producing the best samples.

Training Procedure

Use SGD - algorithm of choice ADAM optimizer on two minibatches simultaneously.

A minibatch of training examples
A minibatch of generated samples Run gradient descent on both of the players cost functions simultaneously