Human action recognition from RGB-D frames

Scopo di questo lavoro è la creazione di un sistema client-server facilmente fruibile per la videosorveglianza tramite la telecamera Omnidome®. Vengono impiegate tecnologie AJAX, PHP, HTML e C++ per la realizzazione di una interfaccia di prenotazione e controllo il più semplice ed intuitiva possibile, con gestione automatizzata della coda di utentiopenEmbargo per motivi di segretezza e di proprietà dei risultati e informazioni sensibil

Video based reconstruction system for mixed reality environments supporting contextualised non-verbal communication and its study

This Thesis presents a system to capture, reconstruct and render the three-dimensional form of people and objects of interest in such detail that the spatial and visual aspects of non-verbal behaviour can be communicated.The system supports live distribution and simultaneous rendering in multiple locations enabling the apparent teleportation of people and objects. Additionally, the system allows for the recording of live sessions and their playback in natural time with free-viewpoint.It utilises components of a video based reconstruction and a distributed video implementation to create an end-to-end system that can operate in real-time and on commodity hardware.The research addresses the specific challenges of spatial and colour calibration, segmentation and overall system architecture to overcome technical barriers, the requirement of domain specific knowledge to setup and generate avatars to a consistent high quality.Applications of the system include, but are not limited to, telepresence, where the computer generated avatars used in Immersive Collaborative Virtual Environments can be replaced with ones that are faithful of the people they represent and supporting researchers in their study of human communication such as gaze, inter-personal distance and facial expression.The system has been adopted in other research projects and is integrated with a mixed reality application where, during a live linkup, a three-dimensional avatar is streamed to multiple end-points across different countries

Vision-Augmented molecular dynamics simulation of nanoindentation

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University London.This thesis has contributed to the literature by providing a pathway to simplify the process of carrying out molecular dynamics simulation. As a part of the investigation, a user-friendly vision-augmented technique was developed to set up and carry out atomistic simulations using hand-gestures. The system is novel in its concept as it enables the user to directly manipulate the atomic structures on the screen, in 3D space using hand gestures, allowing the exploration and visualisation of molecular interactions at different relative conformations. The hand gestures are used to pick and place atoms on the screen allowing thereby the ease of preparing and carrying out molecular dynamics simulations in a more intuitive way. The end result is that users with limited expertise in developing molecular structures can now do so easily and intuitively by the use of body gestures to interact with the simulator to study the system in question. The proposed system was tested by performing parallel molecular dynamics simulations to study (i) crystal anisotropy of a diamond cubic substrate (crystalline silicon) using nanoindentation with a long-range (Screened bond order) Tersoff potential and (ii) crystal anisotropy of a body centre cubic metal (tantalum) using nanoindentation with an Embedded Atomic Method (EAM) type potential. The MD data was post-processed to reveal size effects observed in anisotropy of both these materials, namely, silicon and tantalum. The value of hardness and elastic modulus obtained from the MD data was found in accordance with what has been discovered previously by experiments, thereby validating the simulations. Based on this, it is anticipated that the proposed system will open up new horizons to the current methods on how an MD simulation is designed and executed.King Abdullah bin Abdul-Aziz Al- Saud Ministry of Higher Education and Northern Borders University, Kingdom of Saudi Arabi

Dynamic gesture recognition using transformation invariant hand shape recognition

In this thesis a detailed framework is presented for accurate real time gesture recognition. Our approach to develop a hand-shape classifier, trained using computer animation, along with its application in dynamic gesture recognition is described. The system developed operates in real time and provides accurate gesture recognition. It operates using a single low resolution camera and operates in Matlab on a conventional PC running Windows XP. The hand shape classifier outlined in this thesis uses transformation invariant subspaces created using Principal Component Analysis (PCA). These subspaces are created from a large vocabulary created in a systematic maimer using computer animation. In recognising dynamic gestures we utilise both hand shape and hand position information; these are two o f the main features used by humans in distinguishing gestures. Hidden Markov Models (HMMs) are trained and employed to recognise this combination of hand shape and hand position features. During the course o f this thesis we have described in detail the inspiration and motivation behind our research and its possible applications. In this work our emphasis is on achieving a high speed system that works in real time with high accuracy

Human gait recognition under neutral and non-neutral gait sequences

Rapid advances in biometrics technology makes their use for person‘s identity more acceptable in a variety of applications, especially in the areas of the interest in security and surveillance. The upsurge in terrorist attacks in the past few years has focused research on biometric systems that have the ability to identify individuals from a distance, and this is spearheading research interest in Gait biometric due to being unobtrusive and less dependent on high image/video quality. Gait biometric is a behavioral trait that aims to identify individuals from image sequences based on their walking style. The growing list of possible civil as well as security applications for various purposes is paralleled by the emergence of a variety of research challenges in dealing with a various external as well as internal factors influencing the performance of Gait Recognition (GR) in unconstrained recording conditions. This thesis is concerned with Gait Recognition in unconstrained scenarios aims to address research questions covering (1) The selection of sets of features for a gait signature; (2) The effects of gender and/or recoding condition case (neutral, carrying a bag, coat wearing) on the performance of GR schemes; (3) Integrating gender and/or case classifications into GR; and (4) The role of emerging Kinect sensor technology, with its capability of sensing human skeletal features in GR and applications. Accordingly, our objectives will focus on investigating, developing and testing the performance of using a variety of gait sequencefeatures for the various components/tasks and their integration. Our tests are based on large number of experiments based on CASIA B database as well as an in-house database of Kinect sensor recording. In all experiments, we use different dimension reduction and feature selection methods do reduce the dimensions in these proposed feature vectors, such as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA) and Fisher Score, followed by different classification methods like; k-nearest-neighbour (k-NN), Support Vector Machine (SVM), Naive Bayes and linear discriminant classifier (LDC), to test the performance of the proposed methods. The initial part is focused on reviewing existing background removal for indoor and outdoor scenarios and developing more efficient versions primarily by adopting the work for wavelet domain rather than the traditional spatial domain based schemes. These include motion detection by frame differencing and Mixture of Gaussians, the latter being more reliable for outdoor scenarios. Subsequently, we investigated a variety of features that can be extractedfrom various subbands of wavelet-decomposed frames of different body parts (partitioned according to the golden ratio). We gradually built sets of features, together with their fused combinations, that can categorized as hybrid of model-based and motion-based models. The first list of features developed to deal with Neutral Gait Recognition (NGR) includes: Spatio-Temporal Model (STM), Legs Motion Detection Feature (LMD), and the Statistical model of the approximation LL-wavelet subband images (AWM). We shall demonstrate that fusing these features achieves accuracy of 97%, which is comparable to the state of the art. These features will be shown to achieve 96% accuracy in gender classification (GC), and we shall establish that the NGR2 scheme that integrates GC into NGR improves the accuracy by a noticeable percentage. Testing the performance of these NGR schemes in recognising non-neutral cases revealed the challenges of Unrestricted Gait Recognition (UGR). The second part of the thesis is focused on developing UGR schemes. For this, first a new statistical wavelet feature set extracted from high frequency subbands, called Detail coefficients Wavelet Model (DWM) was added to the previous list. Using different combinations of these schemes, will be shown to significantly improve the performance for non-neutral gait cases, but to less extent in the coat wearing case. We then develop a Gait Sequence Case Detection (GSCD) which has excellent performance. We will show that integrating GSCD and GC together into UGR improves the performance for all cases. We shall also investigate the different UGS scheme that generalizes existing work on Gait Energy and Gait Entropy images (GEI and GEnI) features but in the wavelet domain and in different body parts. Testing these two schemes, and their fusion, post the PCA dimension reduction yield much improved accuracy for the non-neutral cases compared to existing scheme GEI and GEnI schemes, but are significantly outperformed by the last scheme. However, by fusing the UGS scheme with the GSCD+GC+UGR scheme above we will get best accuracy that outperform the state of the art in GR specially in the non-neutral cases. The thesis ended by conducting a rather limited investigation on the use of the Kinect sensors for GR. We develop two sets of features: Horizontal Distance Features and Vertical Distance Features from small set of skeleton point trajectories. The experimental result on neutral was very successful but for the unrestricted gait recognition (with the 5 case variations) satisfactory but not optimal performance relies on the gallery including balanced number of samples from all cases

Self-adaptive structure semi-supervised methods for streamed emblematic gestures

Although many researchers try to improve the level of machine intelligence, there is still a long way to achieve intelligence similar to what humans have. Scientists and engineers are continuously trying to increase the level of smartness of the modern technology, i.e. smartphones and robotics. Humans communicate with each other by using the voice and gestures. Hence, gestures are essential to transfer the information to the partner. To reach a higher level of intelligence, the machine should learn from and react to the human gestures, which mean learning from continuously streamed gestures. This task faces serious challenges since processing streamed data suffers from different problems. Besides the stream data being unlabelled, the stream is long. Furthermore, “concept-drift” and “concept evolution” are the main problems of them. The data of the data streams have several other problems that are worth to be mentioned here, e.g. they are: dynamically changed, presented only once, arrived at high speed, and non-linearly distributed. In addition to the general problems of the data streams, gestures have additional problems. For example, different techniques are required to handle the varieties of gesture types. The available methods solve some of these problems individually, while we present a technique to solve these problems altogether. Unlabelled data may have additional information that describes the labelled data more precisely. Hence, semi-supervised learning is used to handle the labelled and unlabelled data. However, the data size increases continuously, which makes training classifiers so hard. Hence, we integrate the incremental learning technique with semi-supervised learning, which enables the model to update itself on new data without the need of the old data. Additionally, we integrate the incremental class learning within the semi-supervised learning, since there is a high possibility of incoming new concepts in the streamed gestures. Moreover, the system should be able to distinguish among different concepts and also should be able to identify random movements. Hence, we integrate the novelty detection to distinguish between the gestures that belong to the known concepts and those that belong to unknown concepts. The extreme value theory is used for this purpose, which overrides the need of additional labelled data to set the novelty threshold and has several other supportive features. Clustering algorithms are used to distinguish among different new concepts and also to identify random movements. Furthermore, the system should be able to update itself on only the trusty assignments, since updating the classifier on wrongly assigned gesture affects the performance of the system. Hence, we propose confidence measures for the assigned labels. We propose six types of semi-supervised algorithms that depend on different techniques to handle different types of gestures. The proposed classifiers are based on the Parzen window classifier, support vector machine classifier, neural network (extreme learning machine), Polynomial classifier, Mahalanobis classifier, and nearest class mean classifier. All of these classifiers are provided with the mentioned features. Additionally, we submit a wrapper method that uses one of the proposed classifiers or ensemble of them to autonomously issue new labels to the new concepts and update the classifiers on the newly incoming information depending on whether they belong to the known classes or new classes. It can recognise the different novel concepts and also identify random movements. To evaluate the system we acquired gesture data with nine different gesture classes. Each of them represents a different order to the machine e.g. come, go, etc. The data are collected using the Microsoft Kinect sensor. The acquired data contain 2878 gestures achieved by ten volunteers. Different sets of features are computed and used in the evaluation of the system. Additionally, we used real data, synthetic data and public data as support to the evaluation process. All the features, incremental learning, incremental class learning, and novelty detection are evaluated individually. The outputs of the classifiers are compared with the original classifier or with the benchmark classifiers. The results show high performances of the proposed algorithms

Colour and Colorimetry Multidisciplinary Contributions Vol. XIb

It is well known that the subject of colour has an impact on a range of disciplines. Colour has been studied in depth for many centuries, and as well as contributing to theoretical and scientific knowledge, there have been significant developments in applied colour research, which has many implications for the wider socio-economic community. At the 7th Convention of Colorimetry in Parma, on the 1st October 2004, as an evolution of the previous SIOF Group of Colorimetry and Reflectoscopy founded in 1995, the "Gruppo del Colore" was established. The objective was to encourage multi and interdisciplinary collaboration and networking between people in Italy that addresses problems and issues on colour and illumination from a professional, cultural and scientific point of view. On the 16th of September 2011 in Rome, in occasion of the VII Color Conference, the members assembly decided to vote for the autonomy of the group. The autonomy of the Association has been achieved in early 2012. These are the proceedings of the English sessions of the XI Conferenza del Colore