Region-based shape matching for automatic image annotation and query-by-example

Abstract

We present a method for automatic image annotation and retrieval based on query-by-example by region-based shape matching. The proposed method consists of two parts: region selection and shape matching. In the first part, the image is partitioned into disjoint, connected regions with more-or-less uniform color, whose boundaries coincide with spatial edge locations. Each region or valid combinations of neighboring regions constitute “potential objects.” In the second part, the shape of each potential object is tested to determine whether it matches one from a set of given templates. To this effect, we propose a new shape matching method, which is translation-, rotation-, and isotropic scale-invariant, where the boundary of each potential object, as well as of each template, is represented by a B-spline. We, then, identify correspondences between the joint points of the B-splines of potential objects and templates by using a modal matching method. These correspondences are used to estimate the parameters of an affine mapping to register the object with the template. A proximity measure is then computed between the two contours based on the Hausdorff distance. We demonstrate the performance of the proposed method on a variety of images.