Empowering 3D Gaussian Splatting with generalization ability is appealing. However, existing generalizable 3D Gaussian Splatting methods are largely confined to narrow-range interpolation between stereo images due to their heavy backbones, thus lacking the ability to accurately localize 3D Gaussian and support free-view synthesis across wide view range. In this paper, we present a novel framework FreeSplat that is capable of reconstructing geometrically consistent 3D scenes from long sequence input towards free-view synthesis.Specifically, we firstly introduce Low-cost Cross-View Aggregation achieved by constructing adaptive cost volumes among nearby views and aggregating features using a multi-scale structure. Subsequently, we present the Pixel-wise Triplet Fusion to eliminate redundancy of 3D Gaussians in overlapping view regions and to aggregate features observed across multiple views. Additionally, we propose a simple but effective free-view training strategy that ensures robust view synthesis across broader view range regardless of the number of views. Our empirical results demonstrate state-of-the-art novel view synthesis peformances in both novel view rendered color maps quality and depth maps accuracy across different numbers of input views. We also show that FreeSplat performs inference more efficiently and can effectively reduce redundant Gaussians, offering the possibility of feed-forward large scene reconstruction without depth priors.
赋予三维高斯喷溅泛化能力具有吸引力。然而，现有的泛化三维高斯喷溅方法大多限于在立体图像间的狭窄范围内插值，因此缺乏准确定位三维高斯和支持宽视角范围自由视图合成的能力。在本文中，我们提出了一个名为 FreeSplat 的新框架，该框架能够从长序列输入中重建几何上一致的三维场景，以实现自由视图合成。具体来说，我们首先引入了低成本的跨视图聚合，通过在附近视图之间构建自适应的成本体积并使用多尺度结构聚合特征来实现。接下来，我们提出了像素级三元组融合，以消除重叠视图区域中的三维高斯冗余，并聚合跨多个视图观察到的特征。此外，我们提出了一个简单但有效的自由视图训练策略，确保在更广泛的视角范围内实现稳健的视图合成，无论视图数量如何。我们的实验结果显示，在不同输入视图数量下，无论是新视角渲染的色彩地图质量还是深度地图精度，我们的方法都展示了最先进的新视角合成性能。我们还展示了 FreeSplat 在推理上更为高效，并且能有效减少冗余的高斯，提供了无需深度先验的大场景前馈重建的可能性。