Low-powered devices – such as small form factor head-mounted displays (HMDs) – struggle to deliver a smooth and high-quality viewing experience, due to their limited power and rendering capabilities. Cloud rendering attempts to solve the quality issue, but leads to prohibitive latency and bandwidth requirements, hindering use with HMDs over mobile connections or even over Wifi. One solution – split rendering – where frames are partially rendered on the client device, often either requires geometry and rendering hardware, or struggles to generate frames faithfully under viewpoint changes and object motion.
Our method enables spatio-temporal interpolation via bidirectional reprojection to efficiently generate intermediate frames in a split rendering setting, while limiting the communication cost and relying purely on image-based rendering. Furthermore, our method is robust to modest connectivity issues and handles effects such as dynamic smooth shadows.
We extent image based bidirectional reprojection [Yang et. al. 2011] to a dynamic split rendering pipeline that generates high framerate intermediate frames from sparse server frame data. Bidirectional reprojection allows for dynamic scene content. By incorperating object motion it is able to deliver much more stable intermediate frames compared to previous approaches. To address disocclusions, we introduce auxiliary viewpoints that capture additional scene information to fill in missing regions. Furthermore, we integrate a shadow tracking mechanism to ensure accurate rendering of dynamic soft shadows.
@article{steiner2025SplitRendering,
author = {Steiner, Michael and K{\"o}hler, Thomas and Radl, Lukas and Budge, Brian and Steinberger, Markus},
title = {{Image-Based Spatio-Temporal Interpolation for Split Rendering}},
journal = {Computer Graphics Forum},
number = {8},
volume = {44},
year = {2025}
}
[Yang et. al. 2011] Lei Yang, Yu-Chiu Tse, Pedro V. Sander, Jason Lawrence, Diego Nehab, Hugues Hoppe, and Clara L. Wilkins. 2011. Image-based bidirectional scene reprojection. ACM TOG 30(6).