Intuitive, Interactive Beard and Hair Synthesis With Generative Models

Type: Article

Publication Date: 2020-06-01

Citations: 37

DOI: https://doi.org/10.1109/cvpr42600.2020.00747

View Chat PDF

Abstract

We present an interactive approach to synthesizing realistic variations in facial hair in images, ranging from subtle edits to existing hair to the addition of complex and challenging hair in images of clean-shaven subjects. To circumvent the tedious and computationally expensive tasks of modeling, rendering and compositing the 3D geometry of the target hairstyle using the traditional graphics pipeline, we employ a neural network pipeline that synthesizes realistic and detailed images of facial hair directly in the target image in under one second. The synthesis is controlled by simple and sparse guide strokes from the user defining the general structural and color properties of the target hairstyle. We qualitatively and quantitatively evaluate our chosen method compared to several alternative approaches. We show compelling interactive editing results with a prototype user interface that allows novice users to progressively refine the generated image to match their desired hairstyle, and demonstrate that our approach also allows for flexible and high-fidelity scalp hair synthesis.

Locations

  • arXiv (Cornell University) - View - PDF
  • 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) - View

Similar Works

Action Title Year Authors
+ Intuitive, Interactive Beard and Hair Synthesis with Generative Models 2020 Kyle Olszewski
Duygu Ceylan
Jun Xing
Jose Echevarria
Zhili Chen
Weikai Chen
Hao Li
+ Neural Strands: Learning Hair Geometry and Appearance from Multi-View Images 2022 Radu Alexandru Roşu
Shunsuke Saito
Ziyan Wang
Chenglei Wu
Sven Behnke
Giljoo Nam
+ MichiGAN: Multi-Input-Conditioned Hair Image Generation for Portrait Editing 2020 Zhentao Tan
Menglei Chai
Dongdong Chen
Jing Liao
Qi Chu
Lu Yuan
Sergey Tulyakov
Nenghai Yu
+ PDF Chat \textsc{Perm}: A Parametric Representation for Multi-Style 3D Hair Modeling 2024 Chengan He
Xin Sun
Zhixin Shu
Fu-Jun Luan
Sören Pirk
Jorge Alejandro Amador Herrera
Dominik L. Michels
Tuanfeng Y. Wang
Meng Zhang
Holly Rushmeier
+ Barbershop: GAN-based Image Compositing using Segmentation Masks. 2021 Peihao Zhu
Rameen Abdal
John Femiani
Peter Wonka
+ SketchHairSalon: Deep Sketch-based Hair Image Synthesis 2021 Chufeng Xiao
Yu Deng
Xiaoguang Han
Youyi Zheng
Hongbo Fu
+ Text-Guided Generation and Editing of Compositional 3D Avatars 2023 Hao Zhang
Yao Feng
Peter Kulits
Yandong Wen
Justus Thies
Michael J. Black
+ PDF Chat Hair color digitization through imaging and deep inverse graphics 2022 Robin Kips
Panagiotis-Alexandros Bokaris
Matthieu Perrot
Pietro Gori
Isabelle Bloch
+ DeepSketchHair: Deep Sketch-based 3D Hair Modeling 2019 Yuefan Shen
Changgeng Zhang
Hongbo Fu
Kun Zhou
Youyi Zheng
+ Neural Haircut: Prior-Guided Strand-Based Hair Reconstruction 2023 Vanessa Sklyarova
Jenya Chelishev
Andreea Dogaru
Igor Medvedev
Victor Lempitsky
Egor Zakharov
+ DeepSketchHair: Deep Sketch-Based 3D Hair Modeling 2020 Yuefan Shen
Changgeng Zhang
Hongbo Fu
Kun Zhou
Youyi Zheng
+ PDF Chat GroomGen: A High-Quality Generative Hair Model Using Hierarchical Latent Representations 2023 Yuxiao Zhou
Menglei Chai
Alessandro Pepe
Markus Groß
Thabo Beeler
+ Neural Hair Rendering 2020 Menglei Chai
Jian Ren
Sergey Tulyakov
+ Neural Hair Rendering 2020 Menglei Chai
Jian Ren
Sergey Tulyakov
+ HairNet: Single-View Hair Reconstruction using Convolutional Neural Networks 2018 Yi Zhou
Liwen Hu
Jun Xing
Weikai Chen
Han-Wei Kung
Xin Tong
Hao Li
+ Single-View Hair Reconstruction using Convolutional Neural Networks 2018 Yi Zhou
Liwen Hu
Jun Xing
Weikai Chen
Han-Wei Kung
Tong Xin
Hao Li
+ Efficient Hair Style Transfer with Generative Adversarial Networks 2022 Muhammed Pektaş
Barış Geçer
Aybars Uğur
+ PDF Chat HairFastGAN: Realistic and Robust Hair Transfer with a Fast Encoder-Based Approach 2024 Maxim Nikolaev
М. В. Кузнецов
Dmitry Vetrov
Aibek Alanov
+ PDF Chat Head360: Learning a Parametric 3D Full-Head for Free-View Synthesis in 360{\deg} 2024 Yuxiao He
Yiyu Zhuang
Yanwen Wang
Yao Yao
Siyu Zhu
Xiaoyu Li
Qi Zhang
Xun Cao
Hao Zhu
+ PhoMoH: Implicit Photorealistic 3D Models of Human Heads 2022 Mihai Zanfir
Thiemo Alldieck
Cristian Sminchisescu

Cited by (16)

Action Title Year Authors
+ PDF Chat Talk-to-Edit: Fine-Grained Facial Editing via Dialog 2021 Yuming Jiang
Ziqi Huang
Xingang Pan
Chen Change Loy
Ziwei Liu
+ PDF Chat Dual In-painting Model for Unsupervised Gaze Correction and Animation in the Wild 2020 Jichao Zhang
Jingjing Chen
Hao Tang
Wei Wang
Yan Yan
Enver Sangineto
Nicu Sebe
+ PDF Chat NeuWigs: A Neural Dynamic Model for Volumetric Hair Capture and Animation 2023 Ziyan Wang
Giljoo Nam
Tuur Stuyck
Stephen Lombardi
Chen Cao
Jason Saragih
Michael Zollhöfer
Jessica K. Hodgins
Christoph Lassner
+ PDF Chat Adversarial Open Domain Adaptation for Sketch-to-Photo Synthesis 2022 Xiaoyu Xiang
Ding Liu
Xiao Yang
Yiheng Zhu
Xiaohui Shen
Jan P. Allebach
+ PDF Chat Editing Conditional Radiance Fields 2021 Steven Liu
Xiuming Zhang
Zhoutong Zhang
Richard Zhang
Jun-Yan Zhu
Bryan Russell
+ PDF Chat LightPainter: Interactive Portrait Relighting with Freehand Scribble 2023 Yiqun Mei
He Zhang
Xuaner Zhang
Jianming Zhang
Zhixin Shu
Yilin Wang
Zijun Wei
Yan Shi
HyunJoon Jung
Vishal M. Patel
+ Neural Hair Rendering 2020 Menglei Chai
Jian Ren
Sergey Tulyakov
+ Sketch Your Own GAN 2021 Sheng-Yu Wang
David Bau
Jun-Yan Zhu
+ PDF Chat Unsupervised High-Resolution Portrait Gaze Correction and Animation 2022 Jichao Zhang
Jingjing Chen
Hao Tang
Enver Sangineto
Peng Wu
Yan Yan
Nicu Sebe
Wei Wang
+ Editing Conditional Radiance Fields 2021 Steven Liu
Xiuming Zhang
Zhoutong Zhang
Richard Zhang
Jun-Yan Zhu
Bryan Russell
+ Talk-to-Edit: Fine-Grained Facial Editing via Dialog 2021 Yuming Jiang
Ziqi Huang
Xingang Pan
Chen Change Loy
Ziwei Liu
+ SketchHairSalon: Deep Sketch-based Hair Image Synthesis 2021 Chufeng Xiao
Yu Deng
Xiaoguang Han
Youyi Zheng
Hongbo Fu
+ PDF Chat Barbershop 2021 Peihao Zhu
Rameen Abdal
John Femiani
Peter Wonka
+ PDF Chat Neural Strands: Learning Hair Geometry and Appearance from Multi-view Images 2022 Radu Alexandru Roşu
Shunsuke Saito
Ziyan Wang
Chenglei Wu
Sven Behnke
Giljoo Nam
+ Barbershop: GAN-based Image Compositing using Segmentation Masks. 2021 Peihao Zhu
Rameen Abdal
John Femiani
Peter Wonka
+ PDF Chat SPHEAR: Spherical Head Registration for Complete Statistical 3D Modeling 2024 Eduard Gabriel Băzăvan
Andrei Zanfir
Teodor Alexandru Szente
Mihai Zanfir
Thiemo Alldieck
Cristian Sminchisescu

Citing (26)

Action Title Year Authors
+ Very Deep Convolutional Networks for Large-Scale Image Recognition 2014 Karen Simonyan
Andrew Zisserman
+ A Neural Algorithm of Artistic Style 2015 Leon A. Gatys
Alexander S. Ecker
Matthias Bethge
+ PDF Chat Perceptual Losses for Real-Time Style Transfer and Super-Resolution 2016 Justin Johnson
Alexandre Alahi
Li Fei-Fei
+ PDF Chat Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks 2016 Chuan Li
Michael Wand
+ PDF Chat Generative Visual Manipulation on the Natural Image Manifold 2016 Jun-Yan Zhu
Philipp Krähenbühl
Eli Shechtman
Alexei A. Efros
+ Neural Photo Editing with Introspective Adversarial Networks 2016 Andrew Brock
Theodore Lim
Jacob Ritchie
Nick Weston
+ PDF Chat Scribbler: Controlling Deep Image Synthesis with Sketch and Color 2017 Patsorn Sangkloy
Jingwan Lu
Fang Chen
Fisher Yu
James Hays
+ PDF Chat Photorealistic Facial Texture Inference Using Deep Neural Networks 2017 Shunsuke Saito
Lingyu Wei
Liwen Hu
Koki Nagano
Hao Li
+ PDF Chat The iray light transport simulation and rendering system 2017 Alexander Keller
Carsten Wächter
Matthias Raab
Daniel Seibert
Dietger van Antwerpen
Johann Korndörfer
Lutz Kettner
+ PDF Chat Visual attribute transfer through deep image analogy 2017 Jing Liao
Yuan Yao
Lu Yuan
Gang Hua
Sing Bing Kang
+ GANs Trained by a Two Time-Scale Update Rule Converge to a Nash Equilibrium 2017 Martin Heusel
Hubert Ramsauer
Thomas Unterthiner
Bernhard Nessler
Günter Klambauer
Sepp Hochreiter
+ Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks 2015 Emily Denton
Soumith Chintala
Arthur Szlam
Rob Fergus
+ Photorealistic Facial Texture Inference Using Deep Neural Networks 2016 Shunsuke Saito
Lingyu Wei
Liwen Hu
Koki Nagano
Hao Li
+ PDF Chat Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks 2017 Jun-Yan Zhu
Taesung Park
Phillip Isola
Alexei A. Efros
+ Very Deep Convolutional Networks for Large-Scale Image Recognition 2014 Karen Simonyan
Andrew Zisserman
+ PDF Chat Semantic Image Synthesis With Spatially-Adaptive Normalization 2019 Taesung Park
Ming-Yu Liu
Ting-Chun Wang
Jun-Yan Zhu
+ PDF Chat Image-to-Image Translation with Conditional Adversarial Networks 2017 Phillip Isola
Jun-Yan Zhu
Tinghui Zhou
Alexei A. Efros
+ PDF Chat Eye In-painting with Exemplar Generative Adversarial Networks 2018 Brian Dolhansky
Cristian Canton Ferrer
+ PDF Chat Deep Feature Interpolation for Image Content Changes 2017 Paul Upchurch
Jacob R. Gardner
Geoff Pleiss
Robert Pless
Noah Snavely
Kavita Bala
Kilian Q. Weinberger
+ PDF Chat Semantic Image Inpainting with Deep Generative Models 2017 Raymond A. Yeh
Chen Chen
Teck Yian Lim
Alexander G. Schwing
Mark Hasegawa–Johnson
N. Minh
+ GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium 2017 Martin Heusel
Hubert Ramsauer
Thomas Unterthiner
Bernhard Nessler
Sepp Hochreiter
+ PDF Chat StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks 2017 Han Zhang
Tao Xu
Hongsheng Li
Shaoting Zhang
Xiaogang Wang
Xiaolei Huang
Dimitris Metaxas
+ Texture synthesis using convolutional neural networks 2015 Leon A. Gatys
Alexander S. Ecker
Matthias Bethge
+ PDF Chat SC-FEGAN: Face Editing Generative Adversarial Network With User’s Sketch and Color 2019 Youngjoo Jo
Jongyoul Park
+ PDF Chat Artistic Style Transfer for Videos 2016 Manuel Ruder
Alexey Dosovitskiy
Thomas Brox
+ Deep generative image models using a Laplacian pyramid of adversarial networks 2015 Emily Denton
Soumith Chintala
Arthur Szlam
Rob Fergus