패치와 시멘틱 구조 특성을 이용한 비주얼 객체 인식 및 검색

Abstract

Content based image retrieval has emerged as an important field encompassing fields like image processing, computer vision and artificial intelligence. Near the turn of the 21st century researchers finally got convinced that next evolution of systems would need to understand the semantics of an image, not simply the low level underlying computational features i.e., “bridging the semantic gap”. The image retrieval systems need to be more intelligent, to be able to recognize generic objects and visual object classes at the least and also abstract meanings as feelings, in the far run. This can be stated as the dawn of second generation research in Image retrieval. Recognition of a multitude of objects as dogs, cars etc. is an un-noticeable every day activity, hardly considered an achievement of any subtle order. In contrast, it is the ultimate scientific challenge of computer vision. After 40 years of research, robustly identifying the familiar objects (chair, person, pet), scene categories (beach, forest, office), and activity patterns (conversation, dance, picnic) depicted in family pictures, news segments, or feature films is still far beyond the capabilities of today’s vision systems [Preface: Towards Object level Categorization Eds. Ponce J., et. al., 2006]. Visual object class recognition has gradually evolved from structure based approaches to appearance based techniques and presently processes of the human vision are under immense focus. The thesis proposes a new approach to visual object class recognition with an aim to better understanding and exploration of the underlying principles of human vision. The thesis investigates the basic level of semantic structure formation in the human vision inferential processes which is hierarchically combined with other semantic structures to form meanings at an abstract level. This is a micro level approach compared to other approaches considering the whole image structure as a unit or geometric modeling approaches. Using this approach two sets of semantic features have been derived for visual object class recognition. The algorithm uses the hypothesis in line with Gestalt laws of proximity that; in an image, basic semantic structures are formed by line segments (arcs also approximated and broken into smaller line segments based on pixel deviation threshold) which are in close proximity of each other. Based on the notion of proximity a transitive relation is defined, which combines basic micro level semantic structures hierarchically till such a point where a semantic meaning of the structure can be extracted. The algorithm extracts line segments in an image and then forms semantic groups of these line segments based on a minimum distance threshold from each other. The line segment groups so formed can be differentiated from each other, by the number of group members and their geometrical properties. The geometrical properties of these semantic groups are used to generate rotation, translation and scale invariant histograms used as feature vectors for object class recognition tasks in a K-nearest neighbor framework. In the second approach a semantic group based on the proximity distance is clustered and modeled as a graph vertex. The line segments which are common to more than one semantic group are defined as semantic relations between the semantic groups and are modeled as edges of the graph. This way an image object is transformed into a graph using micro level structure formations. Each vertex and edge is labeled using translation, rotation and scale invariant properties of the member segments of each vertex and edge. From a set of training images, a graph model is constructed for visual object class recognition. The graph model is constructed by iteratively combining the training graphs and frequency labeling the vertices and edges. After the combining phase, all the vertices and edges whose repetition frequency is below a threshold are removed. The final graph model consists of the semantic nodes which are highly common in the training images. The recognition is based on graph matching the query image graph and the model graph. The model graph generates a vote for the query and ties are resolved by considering the node frequencies in the query and model graph. The algorithm has been applied to classify 101 object classes at one time. The results have been compared with existing state of the art approaches and are found promising. Results from above approaches show that low level image structure and other features can be used to construct different type of semantic features, which can help a model or a classifier make more intelligent decisions and work more effectively for the task compared to low level features alone. Our experimental results are comparable, or outperform other state-of-the-art approaches. We have also summarized the state-of-the-art at the time this work was finished. We conclude with a discussion about the possible future extensions.