360-degree Video Content Tracking and Editing

Nowadays, 360-degree visual tracking has become increasingly important in 360-degree video image processing technology. Although visual tracking technology in 2D videos has gradually matured, there is no universal method for visual tracking in 360-degree videos that can effectively address image stretching and object deformation caused by the equirectangular representation of 360-degree images. In this thesis, we propose two novel methods for single-object visual tracking in 360-degree videos.

The first method provides a general approach that enables the application of any 2D visual tracking algorithm to 360-degree videos. This method first converts the original equirectangular image of the 360-degree video into a 2D gnomonic image. A 2D tracking algorithm is then applied to the converted image, and the tracking results are remapped back to the 360-degree video's current frame. This approach ensures compatibility with existing 2D algorithms while addressing image distortion caused by the 360-degree image representation.

The second method builds upon the UPDT algorithm, combining it with our first method to create a robust 360-degree visual tracking algorithm. This improved approach introduces specific enhancements tailored to the unique challenges of 360-degree visual tracking and proposes solutions to these issues, improving robustness and efficiency.

In our experiments, we utilize two datasets to evaluate our proposed methods. For the first time, we conduct a quantitative analysis of 360-degree visual tracking algorithms using the OTB-format equirectangular 360-degree dataset, which consists of 21 video sequences. Additionally, we apply the 360VOT dataset to our algorithms for experimentation. The results demonstrate that our proposed methods significantly enhance the success rate and accuracy of 360-degree visual tracking while ensuring their effectiveness and reliability.

Additionally, we explore the application of 360-degree visual tracking methods in editing, enabling the automatic manipulation of moving objects in 360-degree videos.