SPIKE: Look into CycleGAN

[jasozh]

Task at Hand

Look into CycleGAN as a potential replacement for Canny edge detection. Figure out how this would fit into our project.

[DarrenKey]

Summary

CycleGAN proposes a way to match two unpaired and distinct datasets (i.e. landscape photos and Monet paintings) to train a GAN that maps the two datasets.

Pros:

• makes lines smooth, works well, potentially higher quality for end-user use?
• much more interesting to implement

Cons:

• doesn't strictly follow the edges of the input image (could also be a pro? less accurate sketches for a better overall picture)
• needs a diverse dataset and to train the models (pretrained weights not provided, dataset trained on is just faces)
• needs both a dataset of images and a dataset of sketches

Assessment:

Stick to Canny for now, can view CycleGAN in the future? Could be much more work but is more interesting

Can also look into deep-learning based methods for edge detection like TEED, interesting results in Github and pretrained weights provided

https://github.com/xavysp/TEED

https://arxiv.org/pdf/1703.10593.pdf