A Study of Deep Single Sketch-Based Modeling: View/Style Invariance, Sparsity and Latent Space Disentanglement

14 Pages Posted: 3 Jan 2022

See all articles by Yue Zhong

Yue Zhong

University of Surrey

Yulia Gryaditskaya

University of Surrey

Honggang Zhang

affiliation not provided to SSRN

Yi-Zhe Song

University of Surrey

Abstract

Deep image-based modeling has received a lot of attention in recent years. Sketch-based modeling in particular has gained popularity given the ubiquitous nature of touchscreen devices. In this paper, we (i) study and compare diverse single-image reconstruction methods on sketch input, comparing the different 3D shape representations: multi-view, voxel- and point-cloud-based, mesh-based and implicit ones; and (ii) analyze the main challenges and requirements of sketch-based modeling systems. We introduce the regression loss and provide two variants of its formulation for the two most promising 3D shape representations: point clouds and signed distance functions. We show that this loss can increase general reconstruction accuracy, and the view- and style-robustness of the reconstruction methods. Moreover, we demonstrate that this loss can benefit the disentanglement of latent space to view-invariant and view-specific information, resulting in further improved performance. To address the figure-ground ambiguity typical for sparse human sketches, we propose a two-branch architecture that exploits sparse user labeling. We hope that our work will inform future research on sketch-based modeling. We will release our datasets and their splits to establish the first benchmark in sketch- based modeling.

Keywords: Deep sketch-based modeling, Single-view reconstruction

Suggested Citation

Zhong, Yue and Gryaditskaya, Yulia and Zhang, Honggang and Song, Yi-Zhe, A Study of Deep Single Sketch-Based Modeling: View/Style Invariance, Sparsity and Latent Space Disentanglement. Available at SSRN: https://ssrn.com/abstract=3999114 or http://dx.doi.org/10.2139/ssrn.3999114

Yue Zhong

University of Surrey ( email )

Guildford
Guildford, GU2 5XH
United Kingdom

Yulia Gryaditskaya

University of Surrey ( email )

Guildford
Guildford, GU2 5XH
United Kingdom

Honggang Zhang (Contact Author)

affiliation not provided to SSRN ( email )

No Address Available

Yi-Zhe Song

University of Surrey ( email )

Guildford
Guildford, GU2 5XH
United Kingdom

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
120
Abstract Views
287
rank
320,197
PlumX Metrics