Shapes are represented by graphs whose nodes correspond to shape parts, and edges capture neighbor and part-of interactions between the parts. Shape matching is formulated as finding the subgraph isomorphism that minimizes a convexified quadratic cost.

How to categorize images showing very similar object categories? Use class evidence accumulated from all image features, instead of using the traditional approach of voting class decisions made on each individual feature. We mathematically prove that voting class evidence is better than voting class decisions.

Find correspondences between similar objects in images captured under large variations in scale. Photometric and geometric properties of objects may change with scale, and details visible in a high-zoom image may not be visible in a coarser-scale image. Scale invariance is achieved by decoupling the scales of objects from those of scenes, and by down-weighting the contributions of fine-resolution details to matching.

A subcategory may appear in the hierarchical definitions of many parent categories (e.g., "windows" are often shared by "buildings," "houses," "recreational-vehicles," etc.), or in the definition of a unique parent (e.g., "two-humps-on-the-back" of "camels"). Therefore, detections of different subcategories provide different degrees of evidence for category recognition. This is estimated using local learning.

CST is a hierarchy of region adjacency graphs. A region’s neighbors are computed using an extension to regions of the Voronoi diagram for point patterns. The CST model of a category is learned by simultaneously searching for both the most salient regions, and the most salient containment and neighbor relationships of regions across the images.

Given an image of 2.1D texture, learn without any supervision a generative model of the entire (unoccluded) texel, and use this model for texel segmentation. Learning involves concurrent estimation of the texel-subtexel structure, and the pdf's of each texel part from only partially visible texels discovered in the image.

Given an arbitrary (unlabeled) image set, learn the models of all visual categories present, and their inter-category relationships, i.e., their taxonomy. The taxonomy recursively defines categories as spatial configurations of (simpler) subcategories each of which may be shared by many categories.

ICCV '07 Poster

Paper

UIUC Hoofed Animals Dataset

Slides

The hoofed animals dataset contains very similar categories that share a number of similar parts. Each image may contain multiple instances of multiple categories. Animals are articulated, non-rigid objects, appearing at different scales amidst clutter, and may be partially occluded.

The images show homogeneous, frontally viewed, natural, 2.1D textures, where: (1) Texels are only statistically similar to each other; (2) Texel placement is random; (3) Repetition of subtexels define a finer grain texture coexisting with the main texture; (4) Due to texel overlap, texel contours form complex patterns (e.g., several edges meet at one point), and overlapping texels have low contrasts, all of which makes texel segmentation difficult.

Given a set of images containing frequent occurrences of an unknown visual category, learn geometric, photometric and topological properties of regions defining the category. Learning is unsupervised, because the target category is not defined by the user, and whether and where any instances of the category appear in a specific image is not known. In a new image, segment all occurrences of the learned category.

CVPR '06 Slides

Paper

Segment texture images, and cluster the segments to form a region-based vocabulary of texture primitives. Then, for each texture class, learn a tree-structured belief network (TSBN), where nodes represent the vocabulary primitives, and edges, their statistical dependencies. Classify a new texture image using the TSBN.

ICPR '06 Slides

Paper

SIG-09 @ CVPR 09

1st International Workshop on
Stochastic Image Grammars

Topics of interest:
hierarchical models
semantic contexts
compositionality
taxonomies
ontologies
graph matching
etc.

June 21, 2009

1st Sino-USA Summer School in Vision, Learning, and Pattern Recognition

Peking University
Beijing, China
July 20-26, 2009