Learning Silhouette Features for Control of Human Motion

We present a vision-based performance interface for controlling animated human characters. The system interactively combines information about the user's motion contained in silhouettes from three viewpoints with domain knowledge contained in a motion capture database to produce an animation of high quality. Such an interactive system might be useful for authoring, for teleconferencing, or as a control interface for a character in a game. In our implementation, the user performs in front of three video cameras; the resulting silhouettes are used to estimate his orientation and body configuration based on a set of discriminative local features. Those features are selected by a machine-learning algorithm during a preprocessing step. Sequences of motions that approximate the user's actions are extracted from the motion database and scaled in time to match the speed of the user's motion. We use swing dancing, a complex human motion, to demonstrate the effectiveness of our approach. We compare our results to those obtained with a set of global features, Hu moments, and ground truth measurements from a motion capture system.Engineering and Applied Science

Enhancement of the Fynbos Leaf Optical Recognition Application (FLORA-E)

Object perception, classification and similarity discernment are relatively effortless tasks in humans. The exact method by which the brain achieves these is not yet fully understood. Identification, classification and similarity inference are currently nontrivial tasks for machine learning enabled platforms, even more so for ones operating in real time applications. This dissertation conducted research on the use of machine learning algorithms in object identification and classification by designing and developing an artificially intelligent Fynbos Leaf Optical Recognition Application (FLORA) platform. Previous versions of FLORA (versions A through D) were designed to recognise Proteaceae fynbos leaves by extracting six digital morphological features, then using the k-nearest neighbour (k-NN) algorithm for classification, yielding an 86.6% accuracy. The methods utilised in FLORA-A to -D are ineffective when attempting to classify irregular structured objects with high variability, such as stems and leafy stems. A redesign of the classification algorithms in the latest version, FLORA-E, was therefore necessary to cater for irregular fynbos stems. Numerous algorithms and techniques are available that can be used to achieve this objective. Keypoint matching, moments analysis and image hashing are the three techniques which were investigated in this thesis for suitability in achieving fynbos stem and leaf classification. These techniques form active areas of research within the field of image processing and were chosen because of their affine transformation invariance and low computational complexity, making them suitable for real time classification applications. The resulting classification solution, designed from experimentation on the three techniques under investigation, is a keypoint matching – Hu moment hybrid algorithm who`s output is a similarity index (SI) score that is used to return a ranked list of potential matches. The algorithm showed a relatively high degree of match accuracy when run on both regular (leaves) and irregular (stems) objects. The algorithm successfully achieved a top 5 match rate of 76% for stems, 86% for leaves and 81% overall when tested using a database of 24 fynbos species (predominantly from the Proteaceae family), where each species had approximately 50 sample images. Experimental results show that Hu moment and keypoint classifiers are ideal for real time applications because of their fast-matching capabilities. This allowed the resulting hybrid algorithm to achieve a nominal computation time of ~0.78s per sample on the test apparatus setup for this thesis. The scientific objective of this thesis was to build an artificially intelligent platform capable of correctly classifying fynbos flora by conducting research on object identification and classification algorithms. However, the core driving factor is rooted in the need to promote conservation in the Cape Floristic Region (CFR). The FLORA project is an example of how science and technology can be used as effective tools in aiding conservation and environmental awareness efforts. The FLORA platform can also be a useful tool for professional botanists, conservationists and fynbos enthusiasts by giving them access to an indexed and readily available digital catalogue of fynbos species across the CFR

Ranking News-Quality Multimedia

News editors need to find the photos that best illustrate a news piece and fulfill news-media quality standards, while being pressed to also find the most recent photos of live events. Recently, it became common to use social-media content in the context of news media for its unique value in terms of immediacy and quality. Consequently, the amount of images to be considered and filtered through is now too much to be handled by a person. To aid the news editor in this process, we propose a framework designed to deliver high-quality, news-press type photos to the user. The framework, composed of two parts, is based on a ranking algorithm tuned to rank professional media highly and a visual SPAM detection module designed to filter-out low-quality media. The core ranking algorithm is leveraged by aesthetic, social and deep-learning semantic features. Evaluation showed that the proposed framework is effective at finding high-quality photos (true-positive rate) achieving a retrieval MAP of 64.5% and a classification precision of 70%.Comment: To appear in ICMR'1

Fast embedding for image classification & retrieval and its application to the hostel industry

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonContent-based image classification and retrieval are the automatic processes of taking an unseen image input and extracting its features representing the input image. Then, for the classification task, this mathematically measured input is categorized according to established criteria in the server and consequently shows the output as a result. On the other hand, for the retrieval task, the extracted features of an unseen query image are sent to the server to search for the most visually similar images to a given image and retrieve these images as a result. Despite image features could be represented by classical features, artificial intelligence-based features, Convolutional Neural Networks (CNN) to be precise, have become powerful tools in the field. Nonetheless, the high dimensional CNN features have been a challenge in particular for applications on mobile or Internet of Things devices. Therefore, in this thesis, several fast embeddings are explored and proposed to overcome the constraints of low memory, bandwidth, and power. Furthermore, the first hostel image database is created with three datasets, hostel image dataset containing 13,908 interior and exterior images of hostels across the world, and Hostels-900 dataset and Hostels-2K dataset containing 972 images and 2,380 images, respectively, of 20 London hostel buildings. The results demonstrate that the proposed fast embeddings such as the application of GHM-Rand operator, GHM-Fix operator, and binary feature vectors are able to outperform or give competitive results to those state-of-the-art methods with a lot less computational resource. Additionally, the findings from a ten-year literature review of CBIR study in the tourism industry could picturize the relevant research activities in the past decade which are not only beneficial to the hostel industry or tourism sector but also to the computer science and engineering research communities for the potential real-life applications of the existing and developing technologies in the field

PlaceAvoider: Steering First-Person Cameras away from Sensitive Spaces

Abstract—Cameras are now commonplace in our social and computing landscapes and embedded into consumer devices like smartphones and tablets. A new generation of wearable devices (such as Google Glass) will soon make ‘first-person ’ cameras nearly ubiquitous, capturing vast amounts of imagery without deliberate human action. ‘Lifelogging ’ devices and applications will record and share images from people’s daily lives with their social networks. These devices that automatically capture images in the background raise serious privacy concerns, since they are likely to capture deeply private information. Users of these devices need ways to identify and prevent the sharing of sensitive images. As a first step, we introduce PlaceAvoider, a technique for owners of first-person cameras to ‘blacklist ’ sensitive spaces (like bathrooms and bedrooms). PlaceAvoider recognizes images captured in these spaces and flags them for review before the images are made available to applications. PlaceAvoider performs novel image analysis using both fine-grained image features (like specific objects) and coarse-grained, scene-level features (like colors and textures) to classify where a photo was taken. PlaceAvoider combines these features in a probabilistic framework that jointly labels streams of images in order to improve accuracy. We test the technique on five realistic first-person image datasets and show it is robust to blurriness, motion, and occlusion. I

Efficient video identification based on locality sensitive hashing and triangle inequality

Master'sMASTER OF SCIENC

Multimedia: information representation and access

[About the book] Information retrieval (IR) is a complex human activity supported by sophisticated systems. Information science has contributed much to the design and evaluation of previous generations of IR system development and to our general understanding of how such systems should be designed and yet, due to the increasing success and diversity of IR systems, many recent textbooks concentrate on IR systems themselves and ignore the human side of searching for information. This book is the first text to provide an information science perspective on IR