AIR: An Agent for Robust Image Matching and Retrieval

Abstract

This paper presents a novel scheme coined AIR (Agent for Image Recognition), acting as an agent, to oversee the image matching and retrieval processes. Firstly, neighboring keypoints within close spatial proximity are examined and used to hypothesize true keypoint matches. While this approach is robust to noise (e.g. a tree) since spatial relation is considered, missing (undetected) keypoints in one image can also be recovered resulting in more keypoint matches. Secondly, the agent is able to recognize instability of projective transformations in certain cases (e.g. non-planar scenes). The geometric approach is substituted with LIS (Longest Increasing Subsequence) approach which does not require any complex geometric transformations. The effectiveness of AIR is substantiated by an image retrieval experiment which demonstrates that it achieves a twofold increase in true matches and higher matching accuracy when compared to RANSAC homography approach.

Keywords

• Image recognition;Image matching;Image retrieval;Spatial relation approach;longest increasing subsequence

References

1. Agarwala A, Agrawala M, Cohen M, Salesin D, Szeliski R (2006). Photograping long scenes with multi-viewpoint panoramas. ACM Trans. on Graphics (SIGGRAPH). 25(3): 853-861.
2. Brown M, Lowe DG (2003). Recognizing panoramas. Proc. 9th IEEE Int’l Conference on Computer Vision (ICCV). 2: 1218-1225.
3. Brown M, Lowe DG (2007). Automatic panoramic image stitching using invariant features. Int’l Journal of Computer Vision (IJCV). 74(1): 59-73.
4. Baumberg A (2000). Reliable feature matching across widely separated views. Proc. IEEE Computer Society Int’l Conference on Computer Vision and Pattern Recognition (CVPR). 1: 774-781.
5. Goedeme T, Tuytelaars T, Van-Gool L (2004). Fast wide baseline matching for visual navigation. Proc. IEEE Computer Society Int’l Conference on Computer Vision and Pattern Recognition (CVPR). 1: 24-29.
6. Kannala J, Brandt SS (2007). Quasi-dense wide baseline matching using match propagation. Proc. IEEE Computer Society Int’l Conference on Computer Vision and Pattern Recognition (CVPR). 1-8.
7. Lee JA, Yow KC, Chia YS (2009). Robust matching of building facades under large viewpoint changes. Proc. 12th IEEE Int’l Conference on Computer Vision (ICCV). 1258-1264
8. Katare A, Mitra SK, Banerjee A (2007). Content based image retrieval system for multi object images using combined features. Proc. IEEE Computer Society Int’l Conference on Computer Vision and Pattern Recognition (CVPR). 595-599

9. Wang J, Zha H, Cipolla R (2005). Combining interest points and edges for content-based image retrieval. Proc. 12th IEEE Int’l Conference on Image Processing (ICIP). 3: 1256-1259
10. Belongie S, Malik J, Puzicha J (2002). Shape matching and object recognition using shape contexts. IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI). 23(4): 509-522
11. Frome A, Huber D, Kolluri R, Billow T, Malik J (2004). Recognizing objects in range data using regional point descriptors. Proc. 8th European Conference on Computer Vision (ECCV). 3: 224-237
12. Lowe DG (2004). Distinctive image features from scale-invariant keypoints. Int’l Journal of Computer Vision (IJCV). 60(2): 91-110
13. Leordeanu M, Hebert M, Sukthankar R (2007). Beyond local appearance: Category recognition from pairwise interactions of simple features. Proc. IEEE Computer Society Int’l Conference on Computer Vision and Pattern Recognition (CVPR). 1-8
14. Mikolajczyk K, Leibe B, Schiele B (2006). Multiple object class detection with a generative model. Proc. IEEE Computer Society Int’l Conference on Computer Vision and Pattern Recognition (CVPR). 1: 26-36
15. Mutch J, Lowe DG (2006). Multiclass object recognition with sparse, localized features. Proc. IEEE Computer Society Int’l Conference on Computer Vision and Pattern Recognition (CVPR). 1: 11-18.
16. Mikolajczyk K, Schmid C (2005). A performance evaluation of local descriptors. IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI). 27(10): 1615-1630
17. Bay H, Tuytelaars T, Van-Gool L (2006). SURF: Speeded Up Robust Features. Proc. 9th European Conference on Computer Vision (ECCV). 1: 404-417
18. Lowe DG (2001). Local feature view clustering for 3D object recognition. Proc. IEEE Computer Society Int’l Conference on Computer Vision and Pattern Recognition (CVPR). 1: 682-688
19. Se S, Lowe DG, Little J (2002). Global localization using distinctive visual features. Proc. IEEE/RSJ Int’l Conference on Intelligent Robots and Systems (IROS). 226-231
20. Fischler MA., Bolles RC. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Comm. of the ACM. 24 (6). 381-395
21. Hartley RI, Zisserman A (2000). Multiple View Geometry in Computer Vision. Cambridge University Press UK
22. Nister D., Stewenius H (2006). Scale recognition with a vocabulary tree. Proc. IEEE Computer Society Int’l Conference on Computer Vision and Pattern Recognition (CVPR). Vol. 2. pp. 2161-2168
23. Ballard DH (1981). Generalizing the hough transform to detect arbitrary shapes. Pattern Recognition (PR). 13 (2). 111-122
24. Grimson WEL (1990). Object Recognition by Computer: The Role of Geometric Constraints. The MIT Press Cambridge. 263-284
25. Hough PVC (1962). Method and means for recognizing complex patterns. U.S. Patent 3069654
26. Fredman M (1975). On computing the length of longest increasing subsequences. Discrete Mathematics. 11 (1). 29-35
27. Faugeras O (1993). Three-Dimensional Computer Vision: A Geometric Viewpoint. The MIT Press Cambridge
28. Chandrasekhar V, Chen DM, Tsai SS, Cheung NM, Chen H, Takacs G, Reznik Y, Vedantham R, Grzeszczuk R, Bach J, Girod B (2011). The stanford mobile visual search data set. Proc. 2nd Annual ACM SIGMM Conference on Multimedia Systems (MMSys). 117-122