Graph-based Approaches for Automatic Object Segmentation
Dr.Ing. Michael Ying Yang
Leibniz Universität Hannover
Institut für Informationsverarbeitung
Tel.: +49 - (0)511 762 19579
Fax: +49 - (0)511 762 5333
One of the fundamental problems and the great challenges in photogrammetry and computer vision is automatic segmentation of complex content in images and videos, so called object segmentation, which is to produce a binary segmentation, separating foreground objects from their background.
Object segmentation is the basis for many applications: object tracking, object recognition, 3D reconstruction, robot navigation, activity recognition, and video retrieval. In image object segmentation, one has to consider a prohibitive number of possible pixel groupings that separate the foreground from the background. Apart from the difficulties in establishing an effective framework to divide the image pixels into meaningful groups, the notion of foreground often needs to be properly defined by providing either user inputs or object models. In video object segmentation, one has to automatically segment the objects in an unannotated video. This is a challenging task, as local image measurements often provide only a weak cue. Object appearance may significantly change over the video frames due to changes in the camera viewpoint, scene illumination or object deformation. While this can be attempted by analyzing individual image frames independently, video provides rich additional cues beyond a single image. These cues include object motion, temporal continuity, and long-range temporal object interactions.
In this project we will address object segmentation from images and videos. The main contributions are following: (1) The graph-based image segmentation framework will formulate foreground segmentation as finding a subset of superpixels that partitions a graph over superpixels.
Mathematically, it will be formulated as Min-Cut, with a novel cost function that simultaneously minimizes the inter-class similarity while maximizing the intra-class similarity. (2) A fully automatic and bottom-up approach for the combination of object segmentation and motion segmentation, which is formulated as inference in a unified conditional random field (CRF) model. The CRF contains pixel labeling and trajectory clustering in a single energy function, which integrates dense local interaction and sparse global constraints. Object and trajectory will be optimized in the joint space via the space-time CRF.