A video painting is created from the input video by segmenting objects and painterly rendering each object with a distinct painting style. Spatiotemporal coherence of object segmentation enables an efficient control to vary style parameters in 2D space and time. This, in turn, enables many dimensions of artistic expression, such as emphasis or de-emphasis of objects,  abstraction and detailed realism, etc.

Given data with a huge number of irrelevant features (> 10⁶), select relevant features to data classification. We decompose a nonlinear problem into a set of locally linear ones through local learning, and then globally learn feature relevance within the large margin framework. The number of samples required to maintain the same level of learning accuracy grows only logarithmically with respect to the total number of features. There is no feature-selection algorithm with a better sample complexity, reported in the literature.

Given an arbitrary video, segment all distinct, moving and static objects present. We transitively match contours of image regions across the frames such that the resulting tracks are locally smooth. Region contours are matched by a new circular dynamic-time warping (CDTW) algorithm that generalizes DTW to closed contours.

Given an arbitrary image, discover and segment all distinct texture subimages. We use the meanshift to estimate the pdf of texel appearance and the pdf of texel placement properties. The meanshift uses a new, variable-bandwidth, hierarchical kernel that captures the structural properties of texels.

Shapes are represented by graphs whose nodes correspond to shape parts, and edges capture neighbor and part-of interactions between the parts. Shape matching is formulated as finding the subgraph isomorphism that minimizes a convexified quadratic cost.

How to categorize images showing very similar object categories? Use class evidence accumulated from all image features, instead of using the traditional approach of voting class decisions made on each individual feature. We mathematically prove that voting class evidence is better than voting class decisions.

Find correspondences between similar objects in images captured under large variations in scale. Photometric and geometric properties of objects may change with scale, and details visible in a high-zoom image may not be visible in a coarser-scale image. Scale invariance is achieved by decoupling the scales of objects from those of scenes, and by down-weighting the contributions of fine-resolution details to matching.

A subcategory may appear in the hierarchical definitions of many parent categories (e.g., "windows" are often shared by "buildings," "houses," "recreational-vehicles," etc.), or in the definition of a unique parent (e.g., "two-humps-on-the-back" of "camels"). Therefore, detections of different subcategories provide different degrees of evidence for category recognition. This is estimated using local learning.

CST is a hierarchy of region adjacency graphs. A region’s neighbors are computed using an extension to regions of the Voronoi diagram for point patterns. The CST model of a category is learned by simultaneously searching for both the most salient regions, and the most salient containment and neighbor relationships of regions across the images.

Given an image of 2.1D texture, learn without any supervision a generative model of the entire (unoccluded) texel, and use this model for texel segmentation. Learning involves concurrent estimation of the texel-subtexel structure, and the pdf's of each texel part from only partially visible texels discovered in the image.

Given an arbitrary (unlabeled) image set, learn the models of all visual categories present, and their inter-category relationships, i.e., their taxonomy. The taxonomy recursively defines categories as spatial configurations of (simpler) subcategories each of which may be shared by many categories.

ICCV '07 Poster

Paper

UIUC Hoofed Animals Dataset

Slides

The hoofed animals dataset contains very similar categories that share a number of similar parts. Each image may contain multiple instances of multiple categories. Animals are articulated, non-rigid objects, appearing at different scales amidst clutter, and may be partially occluded.

The images show homogeneous, frontally viewed, natural, 2.1D textures, where: (1) Texels are only statistically similar to each other; (2) Texel placement is random; (3) Repetition of subtexels define a finer grain texture coexisting with the main texture; (4) Due to texel overlap, texel contours form complex patterns (e.g., several edges meet at one point), and overlapping texels have low contrasts, all of which makes texel segmentation difficult.

Given a set of images containing frequent occurrences of an unknown visual category, learn geometric, photometric and topological properties of regions defining the category. Learning is unsupervised, because the target category is not defined by the user, and whether and where any instances of the category appear in a specific image is not known. In a new image, segment all occurrences of the learned category.

CVPR '06 Slides

PAMI Paper

IJCV Special Issue on
Stochastic Image Grammars

1st Sino-USA Summer School in Vision, Learning, and Pattern Recognition

Peking University
Beijing, China
July 20-26, 2009
 

SIG-09 @ CVPR 09

1st International Workshop on
Stochastic Image Grammars

Topics of interest:
hierarchical models
semantic contexts
compositionality
taxonomies
ontologies
graph matching
etc.

June 21, 2009