Trademark image retrieval by local features

The challenge of abstract trademark image retrieval as a test of machine vision algorithms has attracted considerable research interest in the past decade. Current operational trademark retrieval systems involve manual annotation of the images (the current ‘gold standard’). Accordingly, current systems require a substantial amount of time and labour to access, and are therefore expensive to operate. This thesis focuses on the development of algorithms that mimic aspects of human visual perception in order to retrieve similar abstract trademark images automatically. A significant category of trademark images are typically highly stylised, comprising a collection of distinctive graphical elements that often include geometric shapes. Therefore, in order to compare the similarity of such images the principal aim of this research has been to develop a method for solving the partial matching and shape perception problem. There are few useful techniques for partial shape matching in the context of trademark retrieval, because those existing techniques tend not to support multicomponent retrieval. When this work was initiated most trademark image retrieval systems represented images by means of global features, which are not suited to solving the partial matching problem. Instead, the author has investigated the use of local image features as a means to finding similarities between trademark images that only partially match in terms of their subcomponents. During the course of this work, it has been established that the Harris and Chabat detectors could potentially perform sufficiently well to serve as the basis for local feature extraction in trademark image retrieval. Early findings in this investigation indicated that the well established SIFT (Scale Invariant Feature Transform) local features, based on the Harris detector, could potentially serve as an adequate underlying local representation for matching trademark images. There are few researchers who have used mechanisms based on human perception for trademark image retrieval, implying that the shape representations utilised in the past to solve this problem do not necessarily reflect the shapes contained in these image, as characterised by human perception. In response, a ii practical approach to trademark image retrieval by perceptual grouping has been developed based on defining meta-features that are calculated from the spatial configurations of SIFT local image features. This new technique measures certain visual properties of the appearance of images containing multiple graphical elements and supports perceptual grouping by exploiting the non-accidental properties of their configuration. Our validation experiments indicated that we were indeed able to capture and quantify the differences in the global arrangement of sub-components evident when comparing stylised images in terms of their visual appearance properties. Such visual appearance properties, measured using 17 of the proposed metafeatures, include relative sub-component proximity, similarity, rotation and symmetry. Similar work on meta-features, based on the above Gestalt proximity, similarity, and simplicity groupings of local features, had not been reported in the current computer vision literature at the time of undertaking this work. We decided to adopted relevance feedback to allow the visual appearance properties of relevant and non-relevant images returned in response to a query to be determined by example. Since limited training data is available when constructing a relevance classifier by means of user supplied relevance feedback, the intrinsically non-parametric machine learning algorithm ID3 (Iterative Dichotomiser 3) was selected to construct decision trees by means of dynamic rule induction. We believe that the above approach to capturing high-level visual concepts, encoded by means of meta-features specified by example through relevance feedback and decision tree classification, to support flexible trademark image retrieval and to be wholly novel. The retrieval performance the above system was compared with two other state-of-the-art image trademark retrieval systems: Artisan developed by Eakins (Eakins et al., 1998) and a system developed by Jiang (Jiang et al., 2006). Using relevance feedback, our system achieves higher average normalised precision than either of the systems developed by Eakins’ or Jiang. However, while our trademark image query and database set is based on an image dataset used by Eakins, we employed different numbers of images. It was not possible to access to the same query set and image database used in the evaluation of Jiang’s trademark iii image retrieval system evaluation. Despite these differences in evaluation methodology, our approach would appear to have the potential to improve retrieval effectiveness

Multi-Object Shape Retrieval Using Curvature Trees

This work presents a geometry-based image retrieval approach for multi-object images. We commence with developing an effective shape matching method for closed boundaries. Then, a structured representation, called curvature tree (CT), is introduced to extend the shape matching approach to handle images containing multiple objects with possible holes. We also propose an algorithm, based on Gestalt principles, to detect and extract high-level boundaries (or envelopes), which may evolve as a result of the spatial arrangement of a group of image objects. At first, a shape retrieval method using triangle-area representation (TAR) is presented for non-rigid shapes with closed boundaries. This representation is effective in capturing both local and global characteristics of a shape, invariant to translation, rotation, scaling and shear, and robust against noise and moderate amounts of occlusion. For matching, two algorithms are introduced. The first algorithm matches concavity maxima points extracted from TAR image obtained by thresholding the TAR. In the second matching algorithm, dynamic space warping (DSW) is employed to search efficiently for the optimal (least cost) correspondence between the points of two shapes. Experimental results using the MPEG-7 CE-1 database of 1400 shapes show the superiority of our method over other recent methods. Then, a geometry-based image retrieval system is developed for multi-object images. We model both shape and topology of image objects including holes using a structured representation called curvature tree (CT). To facilitate shape-based matching, the TAR of each object and hole is stored at the corresponding node in the CT. The similarity between two CTs is measured based on the maximum similarity subtree isomorphism (MSSI) where a one-to-one correspondence is established between the nodes of the two trees. Our matching scheme agrees with many recent findings in psychology about the human perception of multi-object images. Two algorithms are introduced to solve the MSSI problem: an approximate and an exact. Both algorithms have polynomial-time computational complexity and use the DSW as the similarity measure between the attributed nodes. Experiments on a database of 13500 medical images and a database of 1580 logo images have shown the effectiveness of the proposed method. The purpose of the last part is to allow for high-level shape retrieval in multi-object images by detecting and extracting the envelope of high-level object groupings in the image. Motivated by studies in Gestalt theory, a new algorithm for the envelope extraction is proposed that works in two stages. The first stage detects the envelope (if exists) and groups its objects using hierarchical clustering. In the second stage, each grouping is merged using morphological operations and then further refined using concavity tree reconstruction to eliminate odd concavities in the extracted envelope. Experiment on a set of 110 logo images demonstrates the feasibility of our approach

Visual Perception And Gestalt Grouping In The Landscape: Are Gestalt Grouping Principles Reliable Indicators Of Visual Preference?

Landscape visual preference research has indicated many potential indicators of preference; however a comprehensive framework concerning the relationship between visual preference and perception has not been solidified. Gestalt psychology, the predecessor to visual perception, proposes certain visual grouping tendencies to explain how humans perceive the world. This study examines if Gestalt grouping principles are reliable indicators of preference, and if they may be used to develop a broad context for visual assessment. Visual preference for 36 landscape scenes testing the proximity and similarity of landscape elements were ranked one through five by 1,749 Mississippi State University undergraduate, graduate, and faculty members in a web-based preference survey. Using a two-way between groups analysis of variance (ANOVA) to analyze responses, the results indicate that the proximal and similar configuration of landscape elements within a scene does significantly affect visual preference

Visual Perception And Gestalt Grouping In The Landscape: Are Gestalt Grouping Principles Reliable Indicators Of Visual Preference?

Landscape visual preference research has indicated many potential indicators of preference; however a comprehensive framework concerning the relationship between visual preference and perception has not been solidified. Gestalt psychology, the predecessor to visual perception, proposes certain visual grouping tendencies to explain how humans perceive the world. This study examines if Gestalt grouping principles are reliable indicators of preference, and if they may be used to develop a broad context for visual assessment. Visual preference for 36 landscape scenes testing the proximity and similarity of landscape elements were ranked one through five by 1,749 Mississippi State University undergraduate, graduate, and faculty members in a web-based preference survey. Using a two-way between groups analysis of variance (ANOVA) to analyze responses, the results indicate that the proximal and similar configuration of landscape elements within a scene does significantly affect visual preference

Scene Segmentation and Object Classification for Place Recognition

This dissertation tries to solve the place recognition and loop closing problem in a way similar to human visual system. First, a novel image segmentation algorithm is developed. The image segmentation algorithm is based on a Perceptual Organization model, which allows the image segmentation algorithm to ‘perceive’ the special structural relations among the constituent parts of an unknown object and hence to group them together without object-specific knowledge. Then a new object recognition method is developed. Based on the fairly accurate segmentations generated by the image segmentation algorithm, an informative object description that includes not only the appearance (colors and textures), but also the parts layout and shape information is built. Then a novel feature selection algorithm is developed. The feature selection method can select a subset of features that best describes the characteristics of an object class. Classifiers trained with the selected features can classify objects with high accuracy. In next step, a subset of the salient objects in a scene is selected as landmark objects to label the place. The landmark objects are highly distinctive and widely visible. Each landmark object is represented by a list of SIFT descriptors extracted from the object surface. This object representation allows us to reliably recognize an object under certain viewpoint changes. To achieve efficient scene-matching, an indexing structure is developed. Both texture feature and color feature of objects are used as indexing features. The texture feature and the color feature are viewpoint-invariant and hence can be used to effectively find the candidate objects with similar surface characteristics to a query object. Experimental results show that the object-based place recognition and loop detection method can efficiently recognize a place in a large complex outdoor environment

Computer vision reading on stickers and direct part marking on horticultural products : challenges and possible solutions

Traceability of products from production to the consumer has led to a technological advancement in product identification. There has been development from the use of traditional one-dimensional barcodes (EAN-13, Code 128, etc.) to 2D (two-dimensional) barcodes such as QR (Quick Response) and Data Matrix codes. Over the last two decades there has been an increased use of Radio Frequency Identification (RFID) and Direct Part Marking (DPM) using lasers for product identification in agriculture. However, in agriculture there are still considerable challenges to adopting barcodes, RFID and DPM technologies, unlike in industry where these technologies have been very successful. This study was divided into three main objectives. Firstly, determination of the effect of speed, dirt, moisture and bar width on barcode detection was carried out both in the laboratory and a flower producing company, Brandkamp GmbH. This study developed algorithms for automation and detection of Code 128 barcodes under rough production conditions. Secondly, investigations were carried out on the effect of low laser marking energy on barcode size, print growth, colour and contrast on decoding 2D Data Matrix codes printed directly on apples. Three different apple varieties (Golden Delicious, Kanzi and Red Jonaprince) were marked with various levels of energy and different barcode sizes. Image processing using Halcon 11.0.1 (MvTec) was used to evaluate the markings on the apples. Finally, the third objective was to evaluate both algorithms for 1D and 2D barcodes. According to the results, increasing the speed and angle of inclination of the barcode decreased barcode recognition. Also, increasing the dirt on the surface of the barcode resulted in decreasing the successful detection of those barcodes. However, there was 100% detection of the Code 128 barcode at the company’s production speed (0.15 m/s) with the proposed algorithm. Overall, the results from the company showed that the image-based system has a future prospect for automation in horticultural production systems. It overcomes the problem of using laser barcode readers. The results for apples showed that laser energy, barcode size, print growth, type of product, contrast between the markings and the colour of the products, the inertia of the laser system and the days of storage all singularly or in combination with each other influence the readability of laser Data Matrix codes and implementation on apples. There was poor detection of the Data Matrix code on Kanzi and Red Jonaprince due to the poor contrast between the markings on their skins. The proposed algorithm is currently working successfully on Golden Delicious with 100% detection for 10 days using energy 0.108 J mm-2 and a barcode size of 10 × 10 mm2. This shows that there is a future prospect of not only marking barcodes on apples but also on other agricultural products for real time production

A Comprehensive Classification of Business Activities in the Market of Intellectual Property Rights-related Services

Technology and intellectual property markets have witnessed great developments in the last few decades. Due to intellectual property rights gaining more importance and technology companies opening up their innovation processes, a wide range of intellectual property rights related services have emerged in the last two decades. The goal of this research is to develop a comprehensive classification system of intellectual property rights related services (IPSC). The classification is created by applying an ontology engineering process. The IPSC consists of 72 various IPR services divided into six main categories (100 Legal Service; 200 IP Consulting; 300 Matchmaking and Trading; 400 IP Portfolio Processing; 500 IPR-related Financial Service; 600 IPR-related Communication Service). The implications of the thesis are directed to policy makers, technology transfer managers, C-level executives and innovation researchers. The IPSC enables practitioners and researchers to organize industry data that can be thereafter analyzed for better strategy and policy making. In addition, this contributes towards organizing a more transparent and single intellectual property market.:Acknowledgements I Abstract II Contents IV List of Figures VI List of Tables VII 1. Introduction 1 1.1. Introduction to Technology Markets 1 1.2. Explanation of Key Concepts 5 1.3. Research Questions and Goals 9 1.4. Readers Guide 13 2. Literature Review 15 2.1. Intellectual Property Markets State of the Art Review 15 2.2. Ontology Engineering State of the Art Review 22 3. Methodology 26 3.1. Methontology 26 3.2. Planning the IPSC 29 3.3. Specification 30 3.4. Conceptualization 31 3.5. Formalization 32 3.6. Integration 32 3.7. Evaluation 33 3.8. Documentation 33 3.9. Realization and Maintenance 33 4. Data description and collection framework 34 5. Applying Methontology 46 5.1. Knowledge Acquisition and Planning the IPSC 46 5.2. Specification 46 5.3. Conceptualization 47 5.4. Formalization 54 100 Legal Service 56 200 IP Consulting 60 300 Matchmaking and Trading 65 400 IP Portfolio Processing 72 500 IPR-related Financial Service 76 600 IPR-related Communication Service 81 5.5. Integration 86 5.6. Evaluation 95 5.7. Documentation 104 5.8. Realization and Maintenance of the IPSC 106 6. Interview Results and Further Discussions 108 6.1. Implications for Industry 108 6.2. Contributions of the IPSC 110 6.3. Limitations of the IPSC and Future Work 112 7. Conclusions 116 References 120 List of experts interviewed and the date of interview 129 Appendices 13

Exploiting BERT and RoBERTa to Improve Performance for Aspect Based Sentiment Analysis

Sentiment Analysis also known as opinion mining is a type of text research that analyses people’s opinions expressed in written language. Sentiment analysis brings together various research areas such as Natural Language Processing (NLP), Data Mining, and Text Mining, and is fast becoming of major importance to companies and organizations as it is started to incorporate online commerce data for analysis. Often the data on which sentiment analysis is performed will be reviews. The data can range from reviews of a small product to a big multinational corporation. The goal of performing sentiment analysis is to extract information from those reviews to gauge public opinion for market research, monitor brand and product reputation, and understand customer experiences. Reviews written on the online platform are often in the form of free text and they do not have any standard structure. Dealing with unstructured data is a challenging problem. Sentiment analysis can be done at different levels, and the focus of this research is on aspect-level sentiment analysis. In aspect-level sentiment analysis, there are two tasks that need to be addressed. The first task is aspect identification which is the process of discovering those attributes of the object that people are commenting on. These attributes of the object are called aspects. The second task is the sentiment classification of those reviews using these extracted aspects. For the sentiment analysis, transformer-based pre-trained models such as BERT (Bidirectional Encoder Representations from Transformers) and RoBERTa (A robustly optimized BERT) are used in this research as they make use of embedding vector space that is rich in context. The purpose of this research is to propose a framework for extracting the aspects from the data which can be applied to these pre-trained models. For the first part of the experiment, both the BERT and RoBERTa models are developed without the aspect-based approach. For the second part of the experiment, the aspect-based approach is applied to the same models and their results are compared and evaluated against the equivalent models. The experiment results show that aspect-based approach has increased the performance of the models by almost 1% than the traditional models and the BERT model with the aspect-based approach had the highest accuracy and performance among all the models evaluated in this research.

A Phenomenological Approach to the Later Films of Terrence Malick

The cinematic legacy of Terrence Malick, while not settled because the director still lives and makes films, is already a turbulent one. A reclusive philosophy student, Malick’s early output accumulated admiration when Malick disappeared from cinema for twenty years. Like so many great 20th-century artists, including J.D. Salinger and Thomas Pynchon, Malick’s absence grew his legend, so his return was welcomed with anticipation and acclaim. As Malick’s output becomes more frequent, though, some are growing cold to his work, asserting that it is repetitive and pretentious, and borders on self-parody. Still others charge that Malick was only regarded as a genius because his mythic status remained shrouded in mystery. However, I argue Malick’s career turned with the release of his 2011 film The Tree of Life. While a preoccupation with the beauty of nature and the duality of man floods Malick’s previous films, each film from 2011 to present has ventured farther away from traditional narrative structure and the audience’s expectations of contemporary American cinema and closer to a cinematic memoir that blends aesthetic experimentation with a deep interest in the historically-influential philosophical notions of immanence and transcendence. While the philosophy of Malick’s films is recognizably Christian, as many critics and scholars will note, it runs deeper than that. Malick is concerned with the possibility of the human encounter with the sublime to, as Schopenhauer would describe, awaken self-consciousness. However, while Schopenhauer would have self-consciousness liberating itself from the will, Malick’s account of the sublime and human exaltation reaffirms the individual (his will and his intellect, among other things) through self-consciousness that results from a recognition of each individual person as also being a part of the story of humanity. In doing so, Malick’s phenomenology more closely resembles Heidegger’s “fundamental ontology” and conception of Being as “grounded” in, yet distinct from, a being. Understood this way, Malick’s choice to eschew traditional characterization in his films supports their philosophical interests. Likewise, his cinematography and editing patterns evoke the power of cinema to present memory as associated logic and time as free from linearity. My project will also include the study of neurocinematics to explore how Malick’s experimental aesthetics both underline his philosophical ideas and create a divisive experience for the audience. Particular attention will be paid to shot composition, elements of mise-en-scène, and editing techniques, specifically the duration of individual shots and the effect of juxtaposing different scenes together, to create an associative meaning only possible through non-narrative cinema. Finally, I will show how all of this makes for a Romantic humanism, which Harold Bloom would describe as “an attempt to transcend the human without forsaking humanism.” Traditionally, transcendence is understood as that which goes beyond the physical level. For Malick, though, transcendence is an essential part of the human’s experience of the sublime in the natural, physical world -- in a word, “immanence.” Malick, in abstracting the specifics of plot, attempts to compose a cinematic representation of the essence of a human life by creating a highly-formal aesthetic experience which asks the viewer to consider the metaphysical shining through the mundane. Malick should be understood, then, as documenting the American experience through a complex aesthetic representation of being, transcendence, and immanence. In the final analysis, my project will show how Malick’s aesthetic experimentation engages the viewer neurologically in ways that both upset the expectations of narrative cinema and establish its own cinematic grammar. The philosophical concerns of Malick’s films -- namely, explorations of man’s relationship with the divine through an experience with nature, man’s spiritual journey from darkness into light, the fluidity of time and memory, and ontology of the soul -- necessitate a distinct style, one which seeks to represent a convergence of transcendence and immanence