Overview of the Reconstruction Improvement of Our Hierarchy UGP on Large-Scale and Large-Motion Scenarios.
Recent advances in differentiable rendering have significantly improved dynamic street scene reconstruction. However, the complexity of large-scale scenarios and dynamic elements such as vehicles and pedestrians remains a substantial challenge. Existing methods often struggle to scale to large scenes or accurately model arbitrary dynamics.
To address these limitations, we propose Hierarchy UGP, with the key idea of constructing a hierarchical structure consisting of a root level, sub-scenes level, and primitive level, using Unified Gaussian Primitive (UGP) defined in 4D space as the representation. The root level serves as the entry point for the hierarchy. At the sub-scenes level, the scene is spatially divided into multiple sub-scenes, where various elements are extracted. At the primitive level, each element is modeled with UGPs, and its global pose is controlled by a motion prior related to time. This hierarchical design greatly enhances the model's capacity, enabling our method to model large-scale scenes. Additionally, our UGP allows for the reconstruction of arbitrary dynamic elements.
We conducted experiments on Dynamic City, our proprietary large-scale dynamic street scene dataset, as well as the public Waymo dataset. Experimental results demonstrate that our method achieves state-of-the-art performance. We plan to release the accompanying code and the Dynamic City dataset as open resources to further research within the community.
Method Overview. The large-scale dynamic scene is constructed as a hierarchical tree structure, where both static and dynamic elements are represented using Unified Gaussian Primitives, enabling efficient large-scale dynamic scene reconstruction. As shown in (a), the scene hierarchy consists of the Root Level, Sub-scenes Level, and Primitive Level. The Root Level serves as the entry point for managing the entire structure. The scene is spatially divided into multiple sub-scenes at the Sub-scenes Level, which are further categorized into Sky, Background, Rigid Objects, and Non-Rigid Objects. As depicted in (b), at the Primitive Level, each element is modeled using Unified Gaussian Primitives with distinct properties.
@article{shyyql2024HierarchyUGP,
title = {Hierarchy UGP: Hierarchy Unified Gaussian Primitive for Large-Scale Dynamic Scene Reconstruction},
author = {Sun Hongyang and Yang Qinglin and Wang Jiawei and Xu Zhen and Liu Chen and Wang Yida and Zhan Kun and Bao Hujun and Peng Sida and Zhou Xiaowei},
year = {2024},
}