Advancing Animal Shape and Pose Reconstruction from Video: Challenges and Optimization Strategies

The reconstruction of animal shape and pose from video is a challenging problem in computer vision, with many approaches utilizing the SMAL as a prior. A promising refinement in this field involves applying Gaussian Splatting on top of those approaches to enhance the accuracy of these reconstructions. However, our observations show that directly applying Gaussian Splatting to each frame independently does not yield optimal results due to the limited precision of the backbone models. In this talk we will highlight the need for an additional optimization step, potentially using temporal constraints between frames, and present our strategies to achieve more consistent and accurate results.