Gait recognition and understanding based on hierarchical temporal memory using 3D gait semantic folding

Gait recognition and understanding systems have shown a wide-ranging application prospect. However, their use of unstructured data from image and video has affected their performance, e.g., they are easily influenced by multi-views, occlusion, clothes, and object carrying conditions. This paper addresses these problems using a realistic 3-dimensional (3D) human structural data and sequential pattern learning framework with top-down attention modulating mechanism based on Hierarchical Temporal Memory (HTM). First, an accurate 2-dimensional (2D) to 3D human body pose and shape semantic parameters estimation method is proposed, which exploits the advantages of an instance-level body parsing model and a virtual dressing method. Second, by using gait semantic folding, the estimated body parameters are encoded using a sparse 2D matrix to construct the structural gait semantic image. In order to achieve time-based gait recognition, an HTM Network is constructed to obtain the sequence-level gait sparse distribution representations (SL-GSDRs). A top-down attention mechanism is introduced to deal with various conditions including multi-views by refining the SL-GSDRs, according to prior knowledge. The proposed gait learning model not only aids gait recognition tasks to overcome the difficulties in real application scenarios but also provides the structured gait semantic images for visual cognition. Experimental analyses on CMU MoBo, CASIA B, TUM-IITKGP, and KY4D datasets show a significant performance gain in terms of accuracy and robustness

A multilevel paradigm for deep convolutional neural network features selection with an application to human gait recognition

Human gait recognition (HGR) shows high importance in the area of video surveillance due to remote access and security threats. HGR is a technique commonly used for the identification of human style in daily life. However, many typical situations like change of clothes condition and variation in view angles degrade the system performance. Lately, different machine learning (ML) techniques have been introduced for video surveillance which gives promising results among which deep learning (DL) shows best performance in complex scenarios. In this article, an integrated framework is proposed for HGR using deep neural network and fuzzy entropy controlled skewness (FEcS) approach. The proposed technique works in two phases: In the first phase, deep convolutional neural network (DCNN) features are extracted by pre-trained CNN models (VGG19 and AlexNet) and their information is mixed by parallel fusion approach. In the second phase, entropy and skewness vectors are calculated from fused feature vector (FV) to select best subsets of features by suggested FEcS approach. The best subsets of picked features are finally fed to multiple classifiers and finest one is chosen on the basis of accuracy value. The experiments were carried out on four well-known datasets, namely, AVAMVG gait, CASIA A, B and C. The achieved accuracy of each dataset was 99.8, 99.7, 93.3 and 92.2%, respectively. Therefore, the obtained overall recognition results lead to conclude that the proposed system is very promising

Decomposition of 3D joint kinematics of walking in Drosophila melanogaster

Animals exhibit a rich repertoire of locomotive behaviors. In the context of legged locomotion, i.e. walking, animals can change their heading direction, traverse diverse substrates with different speeds, or can even compensate for the loss of a leg. This versatility emerges from the fact that biological limbs have more joints and/or more degrees of freedom (DOF), i.e. independent directions of motions, than required for any single movement task. However, this further entails that multiple, or even infinitely many, joint configuration can result in the same leg stepping pattern during walking. How the nervous system deals with such kinematic redundancy remains still unknown. One proposed hypothesis is that the nervous system does not control individual DOFs, but uses flexible combinations of groups of anatomical or functional DOFs, referred to as motor synergies. Drosophila melanogaster represents an excellent model organism for studying the motor control of walking, not least because of the extensive genetic toolbox available, which, among others, allows the identification and targeted manipulation of individual neurons or muscles. However, their tiny size and ability for relatively rapid leg movements hampered research on the kinematics at the level of leg joints due to technical limitations until recently. Hence, the main objective of this dissertation was to investigate the three-dimensional (3D) leg joint kinematics of Drosophila during straight walking. For this, I first established a motion capture setup for Drosophila which allowed the accurate reconstruction of the leg joint positions in 3D with high temporal resolution (400 Hz). Afterwards, I created a kinematic leg model based on anatomical landmarks, i.e. joint condyles, extracted from micro computed-tomography scan data. This step was essential insofar that the actual DOFs of the leg joints in Drosophila were currently unknown. By using this kinematic model, I have found that a mobile trochanter-femur joint can best explain the leg movements of the front legs, but is not mandatory in the other leg pairs. Additionally, I demonstrate that rotations of the femur-tibia plane in the middle legs arise from interactions between two joints suggesting that the natural orientation of joint rotational axes can extent the leg movement repertoire without increasing the number of elements to be controlled. Furthermore, each leg pair exhibited distinct joint kinematics in terms of the joint DOFs employed and their angle time courses during swing and stance phases. Since it is proposed that the nervous system could use motor synergies to solve the redundancy problem, I finally aimed to identify kinematic synergies based on the obtained joint angles from the kinematic model. By applying principal component analysis on the mean joint angle sets of leg steps, I found that three kinematic synergies are sufficient to reconstruct the movements of the tarsus tip during stepping for all leg pairs. This suggests that the problem of controlling seven to eight joint DOFs can be in principle reduced to three control parameters. In conclusion, this dissertation provides detailed insights into the leg joint kinematics of Drosophila during forward walking which are relevant for deciphering motor control of walking in insects. When combined with the extensive genetic toolbox offered by Drosophila as model organism, the experimental platform presented here, i.e. the 3D motion capture setup and the kinematic leg model, can facilitate investigations of Drosophila walking behavior in the future

Interlandmark measurements from lodox statscan images with application to femoral neck anteversion assessment

Includes abstract.Includes bibliographical references.Clinicians often take measurements between anatomical landmarks on X-ray radiographs for diagnosis and treatment planning, for example in orthopaedics and orthodontics. X-ray images, however, overlap three-dimensional internal structures onto a two-dimensional plane during image formation. Depth information is therefore lost and measurements do not truly reflect spatial relationships. The main aim of this study was to develop an inter-landmark measurement tool for the Lodox Statscan digital radiography system. X-ray stereophotogrammetry was applied to Statscan images to enable three-dimensional point localization for inter-landmark measurement using two-dimensional radiographs. This technique requires images of the anatomical region of interest to be acquired from different perspectives as well as a suitable calibration tool to map image coordinates to real world coordinates. The Statscan is suited to the technique because it is capable of axial rotations for multiview imaging. Three-dimensional coordinate reconstruction and inter-landmark measurements were taken using a planar object and a dry pelvis specimen in order to assess the intra-observer measurement accuracy, reliability and precision. The system yielded average (X, Y, Z) coordinate reconstruction accuracy of (0.08 0.12 0.34) mm and resultant coordinate reconstruction accuracy within 0.4mm (range 0.3mm – 0.6mm). Inter-landmark measurements within 2mm for lengths and 1.80 for angles were obtained, with average accuracies of 0.4mm (range 0.0mm – 2.0 mm) and 0.30 (range 0.0 – 1.8)0 respectively. The results also showed excellent overall precision of (0.5mm, 0.10) and were highly reliable when all landmarks were completely visible in both images. Femoral neck anteversion measurement on Statscan images was also explored using 30 dry right adult femurs. This was done in order to assess the feasibility of the algorithm for a clinical application. For this investigation, four methods were tested to determine the optimal landmarks for measurement and the measurement process involved calculation of virtual landmarks. The method that yielded the best results produced all measurements within 10 of reference values and the measurements were highly reliable with very good precision within 0.10. The average accuracy was within 0.40 (range 0.10 –0.80).In conclusion, X-ray stereophotogrammetry enables accurate, reliable and precise inter-landmark measurements for the Lodox Statscan X-ray imaging system. The machine may therefore be used as an inter-landmark measurement tool for routine clinical applications

Electronic Imaging & the Visual Arts. EVA 2012 Florence

The key aim of this Event is to provide a forum for the user, supplier and scientific research communities to meet and exchange experiences, ideas and plans in the wide area of Culture & Technology. Participants receive up to date news on new EC and international arts computing & telecommunications initiatives as well as on Projects in the visual arts field, in archaeology and history. Working Groups and new Projects are promoted. Scientific and technical demonstrations are presented

Contributions to Robust Multi-view 3D Action Recognition

This thesis focus on human action recognition using volumetric reconstructions obtained from multiple monocular cameras. The problem of action recognition has been addressed using di erent approaches, both in the 2D and 3D domains, and using one or multiple views. However, the development of robust recognition methods, independent from the view employed, remains an open problem. Multi-view approaches allow to exploit 3D information to improve the recognition performance. Nevertheless, manipulating the large amount of information of 3D representations poses a major problem. As a consequence, standard dimensionality reduction techniques must be applied prior to the use of machine learning approaches. The rst contribution of this work is a new descriptor of volumetric information that can be further reduced using standard Dimensionality Reduction techniques in both holistic and sequential recognition approaches. However, the descriptor itself reduces the amount of data up to an order of magnitude (compared to previous descriptors) without a ecting to the classi cation performance. The descriptor represents the volumetric information obtained by SfS techniques. However, this family of techniques are highly in uenced by errors in the segmentation process (e.g., undersegmentation causes false negatives in the reconstructed volumes) so that the recognition performance is highly a ected by this rst step. The second contribution of this work is a new SfS technique (named SfSDS) that employs the Dempster-Shafer theory to fuse evidences provided by multiple cameras. The central idea is to consider the relative position between cameras so as to deal with inconsistent silhouettes and obtain robust volumetric reconstructions. The basic SfS technique still have a main drawback, it requires the whole volume to be analized in order to obtain the reconstruction. On the other hand, octree-based representations allows to save memory and time employing a dynamic tree structure where only occupied nodes are stored. Nevertheless, applying the SfS method to octreebased representations is not straightforward. The nal contribution of this work is a method for generating octrees using our proposed SfSDS technique so as to obtain robust and compact volumetric representations.Esta tesis se centra en el reconocimiento de acciones humanas usando reconstrucciones volum etricas obtenidas a partir de m ultiples c amaras monoculares. El problema del reconocimiento de acciones ha sido tratado usando diferentes enfoques, en los dominios 2D y 3D, y usando una o varias vistas. No obstante, el desarrollo de m etodos de reconocimiento robustos, independientes de la vista empleada, sigue siendo un problema abierto. Los enfoques multi-vista permiten explotar la informaci on 3D para mejorar el rendimiento del reconocimiento. Sin embargo, manipular las grandes cantidades de informaci on de las representaciones 3D plantea un importante problema. Como consecuencia, deben ser aplicadas t ecnicas est andar de reducci on de dimensionalidad con anterioridad al uso de propuestas de aprendizaje. La primera contribuci on de este trabajo es un nuevo descriptor de informaci on volum etrica que puede ser posteriormente reducido mediante t ecnicas est andar de reducci on de dimensionalidad en los enfoques de reconocimiento hol sticos y secuenciales. El descriptor, por si mismo, reduce la cantidad de datos hasta en un orden de magnitud (en comparaci on con descriptores previos) sin afectar al rendimiento de clasi caci on. El descriptor representa la informaci on volum etrica obtenida en t ecnicas SfS. Sin embargo, esta familia de t ecnicas est a altamente in uenciada por los errores en el proceso de segmentaci on (p.e., una sub-segmentaci on causa falsos negativos en los vol umenes reconstruidos) de forma que el rendimiento del reconocimiento est a signi cativamente afectado por este primer paso. La segunda contribuci on de este trabajo es una nueva t ecnica SfS (denominada SfSDS) que emplea la teor a de Dempster-Shafer para fusionar evidencias proporcionadas por m ultiples c amaras. La idea central consiste en considerar la posici on relativa entre c amaras de forma que se traten las inconsistencias en las siluetas y se obtenga reconstrucciones volum etricas robustas. La t ecnica SfS b asica sigue teniendo un inconveniente principal; requiere que el volumen completo sea analizado para obtener la reconstrucci on. Por otro lado, las representaciones basadas en octrees permiten salvar memoria y tiempo empleando una estructura de arbol din amica donde s olo se almacenan los nodos ocupados. No obstante, la aplicaci on del m etodo SfS a representaciones basadas en octrees no es directa. La contribuci on nal de este trabajo es un m etodo para la generaci on de octrees usando nuestra t ecnica SfSDS propuesta de forma que se obtengan representaciones volum etricas robustas y compactas

Humanoid Robots

For many years, the human being has been trying, in all ways, to recreate the complex mechanisms that form the human body. Such task is extremely complicated and the results are not totally satisfactory. However, with increasing technological advances based on theoretical and experimental researches, man gets, in a way, to copy or to imitate some systems of the human body. These researches not only intended to create humanoid robots, great part of them constituting autonomous systems, but also, in some way, to offer a higher knowledge of the systems that form the human body, objectifying possible applications in the technology of rehabilitation of human beings, gathering in a whole studies related not only to Robotics, but also to Biomechanics, Biomimmetics, Cybernetics, among other areas. This book presents a series of researches inspired by this ideal, carried through by various researchers worldwide, looking for to analyze and to discuss diverse subjects related to humanoid robots. The presented contributions explore aspects about robotic hands, learning, language, vision and locomotion