This study addresses the challenge of online 3D model generation for neural rendering using an RGB image stream. Previous research has tackled this issue by incorporating Neural Radiance Fields (NeRF) or 3D Gaussian Splatting (3DGS) as scene representations within dense SLAM methods. However, most studies focus primarily on estimating coarse 3D scenes rather than achieving detailed reconstructions. Moreover, depth estimation based solely on images is often ambiguous, resulting in low-quality 3D models that lead to inaccurate renderings. To overcome these limitations, we propose a novel framework for high-quality 3DGS modeling that leverages an online multi-view stereo (MVS) approach. Our method estimates MVS depth using sequential frames from a local time window and applies comprehensive depth refinement techniques to filter out outliers, enabling accurate initialization of Gaussians in 3DGS. Furthermore, we introduce a parallelized backend module that optimizes the 3DGS model efficiently, ensuring timely updates with each new keyframe. Experimental results demonstrate that our method outperforms state-of-the-art dense SLAM methods, particularly excelling in challenging outdoor environments.
本研究针对使用RGB图像流进行神经渲染的在线3D模型生成这一挑战展开。以往研究通过在密集SLAM方法中引入神经辐射场（NeRF）或三维高斯散射（3DGS）作为场景表示来解决这一问题。然而，大多数研究主要关注粗略3D场景的估计，而非实现细致的重建。此外，仅基于图像的深度估计通常存在歧义，导致3D模型质量较低，从而影响渲染的准确性。 为克服这些局限性，我们提出了一种新的高质量3DGS建模框架，该框架结合了在线多视图立体（MVS）方法。我们的方法使用局部时间窗口中的序列帧估计MVS深度，并应用全面的深度优化技术过滤离群值，从而实现对3DGS中高斯点的精确初始化。此外，我们引入了一个并行化的后端模块，能够高效优化3DGS模型，确保在每个新关键帧加入时及时更新。 实验结果表明，我们的方法在性能上优于现有的最先进密集SLAM方法，特别是在复杂的户外环境中表现突出。