RPG: Learning Recursive Point Cloud Generation

Abstract

In this paper we propose a novel point cloud generator that is able to reconstruct and generate 3D point clouds composed of semantic parts. Given a latent representation of the target 3D model, the generation starts from a single point and gets expanded recursively to produce the high-resolution point cloud via a sequence of point expansion stages. During the recursive procedure of generation, we not only obtain the coarse-to-fine point clouds for the target 3D model from every expansion stage, but also unsupervisedly discover the semantic segmentation of the target model according to the hierarchical/parent-child relation between the points across expansion stages. Moreover, the expansion modules and other elements used in our recursive generator are mostly sharing weights thus making the overall framework light and efficient. Extensive experiments are conducted to demonstrate that our proposed point cloud generator has comparable or even superior performance on both generation and reconstruction tasks in comparison to various baselines, as well as provides the consistent co-segmentation among 3D instances of the same object class.

Experiments

Reconstruction

Examples of reconstruction by our RPG. For each pair, the shape on the left is the input while the output is on the right.

Generation

Qualitative examples of the point clouds generated by our proposed recursive point cloud generator (RPG).

Interpolation

Examples for our interpolation between different shapes: (a) Rows sequentially show the point clouds generated on all the expansion stages while interpolating between the chairs on the bottom-left and bottom-right corners; (b) Each row shows interpolation between two 3D shapes of the same object category; (c) Each row shows interpolation between two shapes from different categories.

Co-segmentation

Visualization of co-segmentation results among object instances from Car, Chair and Airplane categories in ShapeNet. For each object category, the rows sequentially highlight different common parts with green color shared across the instances.

@misc{ko2021rpg,
      title={RPG: Learning Recursive Point Cloud Generation}, 
      author={Wei-Jan Ko and Hui-Yu Huang and Yu-Liang Kuo and Chen-Yi Chiu and Li-Heng Wang and Wei-Chen Chiu},
      year={2021},
      eprint={2105.14322},
      archivePrefix={arXiv},
      primaryClass={cs.CV}}