We compare novel view synthesis results from our method (b) with SaRO-GS (d), focusing on both dynamic regions (e.g., hand movements, cooking actions) and static regions (e.g., bottles, background). The zoomed-in regions (c) highlight that our method produces sharper details and fewer artifacts, demonstrating superior synthesis quality in both motion and static areas. The input images (a) provide context for the scene.
Novel-view synthesis of dynamic scenes, crucial for AR/VR applications, remains a challenging problem. Recent methods adapt representations like 3D Gaussian Splatting (3DGS) and Neural Radiance Fields (NeRF) for dynamic scenes by incorporating time as the fourth dimension (4D Representations). These 4D Representations still suffer from aliasing artifacts, especially when generating novel views from divergent viewpoints (zoom-in/zoom-out opera- tions). While using 3D smoothing filters like those proposed in Mip-Splatting might seem like a possible solution, they fail to account for local motion and also exhibit aliasing. To address this, we propose a motion-aware 3D smooth- ing filter specifically designed for 4D representations. Our approach adapts the filter strength based on local motion information, effectively mitigating aliasing without compro- mising rendering quality. This is achieved by estimating the joint density function for time and focal-to-depth ratio using non-parametric estimation method. During inference, we sample from this joint distribution to derive the appropri- ate smoothing filter. This flexible strategy can be integrated with various 4D representations. Our evaluations on stan- dard datasets demonstrate superior performance compared to state-of-the-art methods.
Method Intuition. Given (a) 4D signal (represented by a 2D signal), different projections of it at timestamps t1, t2 & t3 are represented by a 3D signal (represented by a 1D signal) in (b), (c) and (d). 3D Gaussian primitive which models this continuous 3D signal are observed by multiple cameras and at each time stamp it is band-limited by different sampling intervals.
