6-Degree of Freedom via 360 Panoramic Images

6-Degree of Freedoms (6-DoF) is a concept that refers to the ability to translate and rotate freely in a 3D space. In the field of Virtual Reality (VR), 6-DoF technology enables users to view a captured scene from any angle and position, thus enhancing the immersive experience and facilitating interactive communication in applications such as virtual tourism and teleconferencing. Moreover, the technology can aid in capturing and storing data of a particular area for preserving cultural memory. Due to its versatility and broad applicability, 6-DoF has become a popular research topic in recent years. However, most existing methods approach the task via perspective cameras that would require a time-consuming capture process or specialized capturing rigs that are difficult for casual users to setup.

In this thesis, we approach the 6-DoF problem utilizing 360° panoramic images acquired from a single 360° camera. Our work aims to make the 6-DoF technology more accessible to wider audiences. Initially, we address the challenge of static scenes by proposing a novel method that can generate 6-DoF panoramic images from an unstructured collection of 360° panoramic images, and we present an iterative refinement process that can robustly refine the estimated depth map to a higher quality. Furthermore, we explore a solution for scenes with dynamic objects, where we propose the use of local temporal information to train a deformable neural network that can improve the recovered geometry information, which leads to an enhancement of the final results. Our approach demonstrated how to generate 6-DoF videos with dynamic objects using 360° panoramic images. We show that our method produces superior results with visual and depth quality compared to previous methods.