Publication

Unsupervised Image to Sequence Translation with Canvas-Drawer Networks

AbstractEncoding images as a series of high-level constructs, such as brush strokes or discrete shapes, can often be key to both human and machine understanding. In many cases, however, data is only available in pixel form. We present a method for generating images directly in a high-level domain (e.g. brush strokes), without the need for real pairwise data. Specifically, we train a ”canvas” network to imitate the mapping of high-level constructs to pixels, followed by a high-level ”drawing” network which is optimized through this mapping towards solving a desired image recreation or translation task. We successfully discover sequential vector representations of symbols, large sketches, and 3D objects, utilizing only pixel data. We display applications of our method in image segmentation, and present several ablation studies comparing various configurations.

Download publication

Related Resources

See what’s new.

Publication

2014

Model-based neuroanatomy: Tractography validation, white-matter connections and geometrical organization

Diffusion-weighted MRI (DWI) coupled with fiber tractography allows…

Publication

2012

Housing Agency System: Mass-Customization System for Housing

The field of architecture and houses demands a radical re-invention of…

Publication

2013

Skillometers: Reflective Widgets that Motivate and Help Users to Improve Performance

Applications typically provide ways for expert users to increase their…

Publication

2017

WeBuild: Automatically Distributing Assembly Tasks Among Collocated Workers to Improve Coordination

Physical construction and assembly tasks are often carried out by…

Get in touch

Something pique your interest? Get in touch if you’d like to learn more about Autodesk Research, our projects, people, and potential collaboration opportunities.

Contact us