Abstract

Gaussian Splatting has enabled real-time 3D human avatars with unprecedentedlevels of visual quality. While previous methods require a desktop GPU forreal-time inference of a single avatar, we aim to squeeze multiple Gaussianavatars onto a portable virtual reality headset with real-time drivableinference. We begin by training a previous work, Animatable Gaussians, on ahigh quality dataset captured with 512 cameras. The Gaussians are animated bycontrolling base set of Gaussians with linear blend skinning (LBS) motion andthen further adjusting the Gaussians with a neural network decoder to correcttheir appearance. When deploying the model on a Meta Quest 3 VR headset, wefind two major computational bottlenecks: the decoder and the rendering. Toaccelerate the decoder, we train the Gaussians in UV-space instead ofpixel-space, and we distill the decoder to a single neural network layer.Further, we discover that neighborhoods of Gaussians can share a singlecorrective from the decoder, which provides an additional speedup. Toaccelerate the rendering, we develop a custom pipeline in Vulkan that runs onthe mobile GPU. Putting it all together, we run 3 Gaussian avatars concurrentlyat 72 FPS on a VR headset. Demo videos are athttps://forresti.github.io/squeezeme.