Temporally Consistent Superpixels
Zusammenfassung
This thesis addresses the field of early stage video preprocessing in order to improve and accelerate subsequent processing steps like semantic video segmentation or video-based object tracking. A framework is proposed to segment video streams into temporally consistent superpixels in order to create a representation of the video with far less image primitives than the voxelgrid. The proposed energy-minimization-based approach utilizes a novel hybrid clustering strategy for a multidimensional feature space. Techniques are presented to ensure the consistency of the superpixel flow with the image movement while considering visual occlusion and disocclusion
effects. The effectiveness of the proposed method is shown by a comparison to state-of-theart spatio-temporal oversegmentation algorithms using established benchmark metrics. Additionally, its effectiveness is further demonstrated by showing its application on the real-world scenario of interactive video segmentation.
Kurzfassung
Eine ...
Schlagworte
- Kapitel Ausklappen | EinklappenSeiten
- I–XII
- 1–9 1 Introduction 1–9
- 1.1 Motivation
- 1.2 Related Works
- 1.2.1 Supervoxel Methods
- 1.2.2 Superpixel Methods
- 1.3 Challenges
- 1.4 Contributions
- 1.5 Structure of this Thesis
- 10–33 2 Fundamentals 10–33
- 2.1 Image Segmentation
- 2.1.1 MRF/CRF-based Image Segmentation
- 2.1.2 Segmentation by Clustering
- 2.1.3 SLIC Superpixels
- 2.1.4 Mean Shift Superpixel Segmentation
- 2.2 Optical Flow
- 2.2.1 Horn-Schunck-Method
- 2.2.2 Lucas-Kanade-Method
- 2.2.3 Sparse-to-Dense Optical Flow Conversion
- 2.2.4 Lucas/Kanade meets Horn/Schunck
- 34–45 3 Superpixels with Temporal Consistency 34–45
- 3.1 System Overview
- 3.2 Feature Space Setup
- 3.3 Hybrid Optimization
- 3.4 Sliding Window
- 46–63 4 Superpixel Propagation and Handling of Structural Changes 46–63
- 4.1 Segmentation Propagation
- 4.1.1 Propagation using Dense Optical Flow
- 4.1.2 Efficiency Improvement through Sparse Optical Flow
- 4.2 Handling of Structural Changes
- 4.2.1 Size-based Handling
- 4.2.2 Handling by Occlusion and Disocclusion Detection
- 64–104 5 Experimental Results 64–104
- 5.1 Evaluation of Temporally Consistent Superpixels
- 5.1.1 Benchmark Metrics
- 5.1.2 Data Sets and Experimental Setup
- 5.1.3 Per Frame Segmentation Quality
- 5.1.4 Spatio-Temporal Segmentation Quality
- 5.1.5 Superpixel Label Consistency
- 5.1.6 Complexity Considerations
- 5.2 Demonstration: Interactive Video Segmentation
- 5.2.1 Interactive Video Segmentation using Superpixels
- 5.2.2 Segmentation Quality and Runtime Evaluation
- 105–108 6 Conclusions 105–108
- 109–122 Bibliography 109–122
