While neural rendering has led to impressive advances in scene reconstruction and novel view synthesis, it relies heavily on accurately pre-computed camera poses. To relax this constraint, multiple efforts have been made to train Neural Radiance Fields (NeRFs) without pre-processed camera poses. However, the implicit representations of NeRFs provide extra challenges to optimize the 3D structure and camera poses at the same time. On the other hand, the recently proposed 3D Gaussian Splatting provides new opportunities given its explicit point cloud representations. This paper leverages both the explicit geometric representation and the continuity of the input video stream to perform novel view synthesis without any SfM preprocessing. We process the input frames in a sequential manner and progressively grow the 3D Gaussians set by taking one input frame at a time, without the need to pre-compute the camera poses. Our method significantly improves over previous approaches in view synthesis and camera pose estimation under large motion changes.
虽然神经渲染在场景重建和新视角合成方面取得了令人印象深刻的进展，但它严重依赖于精确预计算的摄像机姿态。为了放宽这一限制，已经做出了多种努力，以在没有预处理摄像机姿态的情况下训练神经辐射场（NeRFs）。然而，NeRFs的隐式表示为同时优化3D结构和摄像机姿态带来了额外的挑战。另一方面，最近提出的3D高斯喷溅由于其明确的点云表示，提供了新的机会。本文利用明确的几何表示和输入视频流的连续性，无需进行任何SfM预处理，就可以进行新视角合成。我们以序列方式处理输入帧，并通过一次处理一个输入帧，逐步增长3D高斯集，无需预先计算摄像机姿态。我们的方法在大幅运动变化下的视图合成和摄像机姿态估计方面，显著改善了以前的方法。