3D shape generation is widely applied in various industries to create, visualize, and analyse complex data, designs, and simulations. Typically, 3D shape generation uses a large dataset of 3D shapes as the input. This paper proposes a variational autoencoder with a signed distance function relativistic average generative adversarial network, referred to as 3D-VAE-SDFRaGAN, for 3D shape generation from 2D input images. Both the generative adversarial network (GAN) and variational autoencoder (VAE) algorithms are typical algorithms used to generate realistic 3D shapes. However, it is very challenging to train a stable 3D shape generation model using VAE-GAN. This paper proposes an efficient approach to stabilize the training process of VAE-GAN to generate high-quality 3D shapes. A 3D mesh-based shape is first generated using a 3D signed distance function representation by feeding a single 2D image into a 3D-VAE-SDFRaGAN network. The signed distance function is used to maintain inside–outside information in the implicit surface representation. In addition, a relativistic average discriminator loss function is employed as the training loss function. The polygon mesh surfaces are then produced via the marching cubes algorithm. The proposed 3D-VAE-SDFRaGAN is evaluated with the ShapeNet dataset. The experimental results indicate a notable enhancement in the qualitative performance, as evidenced by the visual comparison of the generated samples, as well as the quantitative performance evaluation using the chamfer distance metric. The proposed approach achieves an average chamfer distance score of 0.578, demonstrating superior performance compared to existing state-of-the-art models.
25 August 2020 in International Journal of Emerging Trends in Engineering Research
Mesh-based 3-dimensional (3D) shape generation from a 2-dimensional (2D) image using a convolution neural network (CNN) framework is an open problem in the computer graphics and vision domains. Most existing CNN-based frameworks lack robust algorithms that can scale well without combining different shape parts. Also, most CNN-based algorithms lack suitable 3D data representations that can fit into CNN without modification(s) to produce high-quality 3D shapes. This paper presents an approach that integrates a variational autoencoder (VAE) and a generative adversarial network (GAN) called 3 dimensional variational autoencoder signed distance function generative adversarial network (3D-VAE-SDFGAN) to create a 3D shape from a 2D image that considerably improves scalability and visual quality. The proposed method only feeds a single 2D image into the network to produce a mesh-based 3D shape. The network encodes a 2D image of the 3D object into the latent representations, and implicit surface representations of 3D objects corresponding to those 2D images are subsequently generated. Hence, a signed distance function (SDF) is proposed to maintain object inside-outside information in the implicit surface representation. Polygon mesh surfaces are then produced using the marching cubes algorithm. The ShapeNet dataset was used in the experiments to evaluate the proposed 3D-VAE-SDFGAN. The experimental results show that 3D-VAE-SDFGAN outperforms other state-of-the-art models.