Zwicker Part of Team Presenting New Tool for Sketching Faces

A University of Maryland expert in computer graphics is part of a team that designed an advanced face image editing program which uses machine-learning to intuitively produce high-quality, realistic results.

Matthias Zwicker, the Reginald Allan Hahne Endowed E-Innovate Professor in Computer Science, helped write the paper “FaceShop: Deep Sketch-based Face Image Editing” with researchers from the University of Bern in Switzerland.

Zwicker also has an appointment in the University of Maryland Institute for Advanced Computer Studies (UMIACS), where his research focuses on 3–D computer graphics, data-driven modeling and animation, image restoration and de-nosing, and the use of deep learning algorithms for computer visualization.

Zwicker and his team will present their work at SIGGRAPH 2018, held 12–16 August in Vancouver, British Columbia. This annual gathering showcases the world's leading professionals, academics, and creative minds at the forefront of computer graphics and interactive techniques.

To the best of the researchers’ knowledge, they are the first to train the software’s neural network on two simultaneous tasks—image completion and image translation—and combine them into a unified framework to edit images.

“Conventional image editing software assists users to achieve a desired effect by essentially painting each pixel with the required color one-by-one, which can be quite tedious,” says Zwicker. “In contrast, our face editing system enables users to easily express high level goals, such as ‘make the mouth smile,’ ‘open the eyes,’ or ‘make the face rounder’ using simple sketching. The system then automatically fills in all pixel colors, and it makes the output look like a photo-realistic face image using artificial intelligence techniques.”

Not only is their software more efficient and intuitive, it’s also flexible.

"Most other approaches rely on more traditional, handcrafted techniques, which impose some limitations,” says Tiziano Protenier, lead author of the work and Ph.D candidate at the University of Bern. "In contrast, our system is very flexible and allows untrained users to perform complex edits within minutes using an intuitive interface."

Although this software is currently only for editing faces, Zwicker sees the algorithm being further developed to eventually manipulate all types of images.

“In the future, with more training data and larger neural networks, it will likely become possible to edit arbitrary images and still obtain photo-realistic results,” he says. “Such an algorithm could be very useful in a general photo editing system like Photoshop, where it would allow users to manipulate images much more easily and quickly than currently possible.”

—Story by Maria Herd