We compare novel view synthesis results from our method (b) with SaRO-GS (d), focusing on both dynamic regions (e.g., hand movements, cooking actions) and static regions (e.g., bottles, background). The zoomed-in regions (c) highlight that our method produces sharper details and fewer artifacts, demonstrating superior synthesis quality in both motion and static areas. The input images (a) provide context for the scene.
Novel-view synthesis of dynamic scenes, crucial for AR/VR applications, remains a challenging problem. Recent methods adapt representations like 3D Gaussian Splatting (3DGS) and Neural Radiance Fields (NeRF) for dynamic scenes by incorporating time as the fourth dimension (4D representations). These 4D representations still suffer from aliasing artifacts, especially when generating novel views from divergent viewpoints (zoom-in/zoom-out operations). While using 3D smoothing filters like those proposed in Mip-Splatting might seem like a possible solution, they fail to account for local motion and also exhibit aliasing. To address this, we propose a motion-aware 3D smoothing filter specifically designed for 4D representations. Our approach adapts the filter strength based on local motion information, effectively mitigating aliasing without compromising rendering quality. This is achieved by estimating the joint density function of time and focal-to-depth ratio using a non-parametric estimation method. During inference, we sample from this joint distribution to determine the appropriate smoothing filter. This flexible strategy can be integrated with various 4D representations. Our evaluations on standard datasets demonstrate superior performance compared to state-of-the-art methods.
Method Intuition. Given (a) 4D signal (represented by a 2D signal), different projections of it at timestamps t1, t2 & t3 are represented by a 3D signal (represented by a 1D signal) in (b), (c) and (d). 3D Gaussian primitive which models this continuous 3D signal are observed by multiple cameras and at each time stamp it is band-limited by different sampling intervals.
