Leung and Malik (1999)[2] use a discriminative model to compute image elements by clustering the filter responses. At each pixel, a pyramid of image filters at various scales and orientations are convolved with the image as illustrated in Figure 1. The the filter responses are clustered by a K-mean clustering method and the cluster centers are defined as textons. However, the transformations are not considered in this model. The potentially same image structure appears multiple times which are shifted, rotated, or scaled versions of each other.
Figure 1. At each pixel, a pyramid of filters at various scales and orientations are convolved with the image to extract a feature vector.
We[4] adopt a TCA method to overcome the obvious problem by introducing a transformation as hidden (latent) variable. The filters uses in the experiments are shown in Figure 2. Since we know the scales and orientations of the filters, the transformations (scaling and rotation) of feature vectors could be inferred to form tighter clusters as illustrated in figure 3. Thus the potentially same image structures are transformed and thus combined into one cluster.
Figure 2. Filters used in the experiments
Figure 3. The transformations (scaling and rotation) of filter responses.
The TCA filtering idea is tested on a set of images and more meaningful results are obtained. Figure 4 shows two or three textons for each pattern. Click 'more...' for more results.
| Original Image |
cluster 1 |
cluster 2 | cluster 3 | Inversed image patch | more results |
 |
 |
 |
 |
 |
more... |
 |
 |
 |
 |
 |
more... |
 |
 |
 |
 |
more... | |
 |
 |
 |
 |
 |
more... |
 |
 |
 |
 |
 |
more... |
 |
 |
 |
 |
 |
more... |
Figure 4. Two or three textons discovered by TCA filtering method. Click 'more...' for more results.
References:
[1] Guo, C., Zhu, S. and Wu, Y. "Visual learning by integrating descriptive and generative methods", Proc. of 8th Int'l Conf. on Computer Vision, Vancouver, Canada, July 2001
[2] Leung, T. and Malik, J. "Recognizing surface using three-dimensional textons", Proc. of 7th Int'l Conf. on Computer Vision, Corfu, Greece, September 1999
[3] Olshausen, B. and Field, D. "Sparse coding with an over-complete basis set: A strategy employed by V1?", Vision Research, 37:3311-3325, 1997
[4] Zhu, S., Guo, C., Wu, Y. and Wang, Y. "What are Textons?", Proc. of 7th European Conf. on Computer Vision, Copenhagen, Denmark, May-June 2002