SteadySplats: Resampling of Low-Variance Gaussians for High-Fidelity Stochastic Rendering

Stochastic rendering draws each Gaussian along a view ray with probability equal to its blending weight, so the noise of a single sample is the variance of the colors along that ray. Standard 3DGS training, however, matches the ground truth by blending Gaussians of disparate colors. We therefore add a color variance term L_var to the training loss, which encourages all Gaussians that splat onto the same pixel to share the same color, without any change to the renderer.

Each pixel shows the color of a single Gaussian, drawn with probability equal to its blending weight; a new sample is drawn every second. The left half shows a model trained without regularization (λ_var = 0), the right half one trained with λ_var = 2. Hover over or tap the image to inspect a pixel: the side panels show its color variance L_var, the probability of each Gaussian's color and the converged color. Click to lock a pixel, drag the divider to compare, and press Space to pause.

Neighboring pixels see similar color distributions, so each sample can also serve its neighbors. Plain averaging blurs the image; instead, we weight every reused sample by how likely it is at the target pixel relative to its source pixel, which keeps the introduced bias minimal. The occlusion part of these weights would normally require sorting. Since a model trained with L_var has very few relevant occluders per pixel, we approximate it from a small buffer of the K most recent samples at each pixel.

Two neighboring pixels A and B usually see similar, but not identical color distributions (\(p_A \approx p_B\)). With one sample per pixel, both are noisy. With resampling, each pixel also reuses its neighbor's sample, weighted by how likely that sample is at the pixel compared to the neighbor (\(w_{B \rightarrow A} = p_A(s_B) / p_B(s_B)\)). Each pixel thus averages two samples instead of one, while samples that are unlikely at the pixel, e.g. from across an edge, get low weights and do not blur it.

Garden rendered stochastically with one sample per pixel over 32 frames. Hover over or tap the image to inspect a pixel: the panels show the sample it draws in the current frame, the K = 4 past samples in its K-buffer, and how both set the weights with which its 4 neighbors reuse the sample. Click to lock a pixel, step through the frames with the slider or the arrow keys, and press Space to play.

Accumulating samples over frames, resampling improves the image right away: with a single sample per pixel, it gains 4.8 dB PSNR over plain averaging. A uniform cross-shaped blur removes noise similarly at first, but plateaus at around 29.5 dB, as it blends across depth edges, while our depth- and transmittance-aware weights keep edges sharp. Select a sample count in the plot to compare the images, and drag the dividers.

Variance regularization and resampling reinforce each other: with L_var, fewer Gaussians contribute to each pixel, so the occluders stored in the K-buffer match those of the neighboring pixels more closely and the resampling weights become more accurate. On these views, the combination improves the PSNR of a single sample per pixel by 7.2 to 9.7 dB over plain stochastic rendering. Drag the center handle to compare the four variants.

Color distributions are also coherent over time. Under mild assumptions, full spatio-temporal resampling reduces to a recursive exponential moving average along the motion trajectory. Instead of clamping the history color or resetting it at depth changes, we lower the weight of outlier history, which avoids both ghosting and popping.

Compare our full method (left) with StochasticSplats, Gaussian Point Splatting and sorted 3DGS (right) along camera paths. All stochastic methods render one sample per pixel and frame and accumulate the samples over time, each under their respective temporal filter. Drag the vertical divider to compare, and use the buttons to switch the method and the scene.

Our renderer keeps a Gaussian at a pixel if a random threshold, drawn anew each frame, lies below its opacity, and the closest kept Gaussian colors the pixel. With white noise thresholds, the kept samples form clumps and holes, while spatio-temporal blue noise spreads them evenly over neighboring pixels and over frames. A single frame has the same error either way, but the evenly spread noise looks much smoother, and averaging over frames, as our temporal reuse does, removes it much faster. Sharing one blue noise texture between all Gaussians would correlate overlapping ones: the teal Gaussian is then only kept where the orange one in front of it is kept as well, so it never shows through. We therefore offset the texture per primitive.

We evaluate on 21 camera paths across 7 Mip-NeRF 360 scenes (3,152 frames) against sorted α-blending as ground truth, measuring image quality and temporal stability (tPSNR). The best value is shown in bold on dark green, the second best on light green.