Recurrent Attention Models for Depth-Based Person Identification

We present an attention-based model that reasons on human body shape and motion dynamics to identify individuals in the absence of RGB information, hence in the dark. Our approach leverages unique 4D spatio-temporal signatures to address the identification problem across days. Formulated as a reinforcement learning task, our model is based on a combination of convolutional and recurrent neural networks with the goal of identifying small, discriminative regions indicative of human identity. We demonstrate that our model produces state-of-the-art results on several published datasets given only depth images. We further study the robustness of our model towards viewpoint, appearance, and volumetric changes. Finally, we share insights gleaned from interpretable 2D, 3D, and 4D visualizations of our model's spatio-temporal attention.Comment: Computer Vision and Pattern Recognition (CVPR) 201

UNERTAN SYNDROME: A CASE SERIES DEMONSTRATING HUMAN DEVOLUTION

A large family with six individuals exhibiting the Unertan syndrome (UTS)was identified residing in southern Turkey. All of the individuals had mental impairments and walked on all four extremities. The practice of intra-familial\ud marriages suggested that theUTS may be an autosomal recessive disorder, similar to previously described cases. The inferior portions of the cerebellum and vermis were\ud absent as evidenced by MRI and CT scans. The height and head circumference of those affected were within normal ranges. Barany’s test suggested normal vestibular\ud system function. The subjects could not name objects or their close relatives. The males (n = 4) could understand simple questions and commands, but answered questions with only one or two sounds. The females (n = 2) were superior to\ud the males with respect to language skills and walking, suggesting an association between walking and speaking abilities. One male exhibited three walking patterns\ud at the same time: quadripedal, tiptoe, and scissor walking. Another male used two walking styles: quadripedal and toe-walking. It is emphasized that there are important differences between the UTS and the disequilibrium syndrome. It is suggested that the inability to walk upright in those affected with the UTS may be\ud best explained by a disturbance in lateral-balance mechanisms,without being related to the cerebello-vestibular system.An interruption of locomotor development during the transition from quadripedality to bipedality may result in habitual walking on all four extremities and is normal in some children. Because quadripedal\ud gait is an ancestral trait, individuals with the UTS, exhibiting a manifestation of reverse evolution in humans, may be considered an experiment of nature, useful\ud in understanding the mechanisms underlying the transition from quadripedality to bipedality during human evolution. The proposed mutant gene or gene pool playing\ud a role in human quadrupedality may also be responsible for human bipedality at the same time. Herein there is no intent to insult or injure; rather, this report is an\ud endeavor to better understand human beings. Supplementary materials are available for this article. Go to the publisher’s online edition of International Journal of\ud Neuroscience for the following free supplemental resource(s): video clips

UNERTAN SYNDROME: A CASE SERIES DEMONSTRTAING HUMAN DEVOLUTION

A large family with six individuals exhibiting the Unertan syndrome (UTS) was identified residing in southern Turkey. All of the individuals had mental impairments and walked on all four extremities. The intra-familial marriages suggested that the UTS is an autosomal recessive disorder. The inferior portions of the cerebellum and vermis were absent as evidenced by MRI and CT scans. The height and head circumference of those affected were within normal ranges. Barany’s test suggested normal vestibular system function. The subjects could not name objects or their close relatives. The males (n = 4) could understand simple questions, answering them with only one or two sounds. The females (n = 2) were superior to the males with respect to language skills and walking, suggesting an association between walking and speaking abilities. One male exhibited three walking patterns at the same time: quadripedal, tiptoe, and scissor walking. Another male used two walking styles: quadripedal and toe-walking. It is emphasized that there are important differences between the UTS and the disequilibrium syndrome (DES). It is suggested that the inability to walk upright in those affected with the UTS may be best explained by a disturbance in lateral-balance mechanisms. An interruption of locomotor development during the transition from quadripedality to bipedality may result in habitual walking on all four extremities and is normal in some children. Since quadripedal gait is an ancestral trait, individuals with the UTS, exhibiting a manifestation of reverse evolution in humans, may be considered an experiment of nature, useful in understanding the mechanisms underlying the transition from quadripedality to bipedality during human evolution. The proposed mutant gene or gene pool playing a role in human quadrupedality may also be responsible for human bipedality at the same time. Herein there is no intent to insult or injure, rather this report is an endeavor to better understand human beings

Gait recognition and understanding based on hierarchical temporal memory using 3D gait semantic folding

Gait recognition and understanding systems have shown a wide-ranging application prospect. However, their use of unstructured data from image and video has affected their performance, e.g., they are easily influenced by multi-views, occlusion, clothes, and object carrying conditions. This paper addresses these problems using a realistic 3-dimensional (3D) human structural data and sequential pattern learning framework with top-down attention modulating mechanism based on Hierarchical Temporal Memory (HTM). First, an accurate 2-dimensional (2D) to 3D human body pose and shape semantic parameters estimation method is proposed, which exploits the advantages of an instance-level body parsing model and a virtual dressing method. Second, by using gait semantic folding, the estimated body parameters are encoded using a sparse 2D matrix to construct the structural gait semantic image. In order to achieve time-based gait recognition, an HTM Network is constructed to obtain the sequence-level gait sparse distribution representations (SL-GSDRs). A top-down attention mechanism is introduced to deal with various conditions including multi-views by refining the SL-GSDRs, according to prior knowledge. The proposed gait learning model not only aids gait recognition tasks to overcome the difficulties in real application scenarios but also provides the structured gait semantic images for visual cognition. Experimental analyses on CMU MoBo, CASIA B, TUM-IITKGP, and KY4D datasets show a significant performance gain in terms of accuracy and robustness

Multi-set canonical correlation analysis for 3D abnormal gait behaviour recognition based on virtual sample generation

Small sample dataset and two-dimensional (2D) approach are challenges to vision-based abnormal gait behaviour recognition (AGBR). The lack of three-dimensional (3D) structure of the human body causes 2D based methods to be limited in abnormal gait virtual sample generation (VSG). In this paper, 3D AGBR based on VSG and multi-set canonical correlation analysis (3D-AGRBMCCA) is proposed. First, the unstructured point cloud data of gait are obtained by using a structured light sensor. A 3D parametric body model is then deformed to fit the point cloud data, both in shape and posture. The features of point cloud data are then converted to a high-level structured representation of the body. The parametric body model is used for VSG based on the estimated body pose and shape data. Symmetry virtual samples, pose-perturbation virtual samples and various body-shape virtual samples with multi-views are generated to extend the training samples. The spatial-temporal features of the abnormal gait behaviour from different views, body pose and shape parameters are then extracted by convolutional neural network based Long Short-Term Memory model network. These are projected onto a uniform pattern space using deep learning based multi-set canonical correlation analysis. Experiments on four publicly available datasets show the proposed system performs well under various conditions

RGBD Datasets: Past, Present and Future

Since the launch of the Microsoft Kinect, scores of RGBD datasets have been released. These have propelled advances in areas from reconstruction to gesture recognition. In this paper we explore the field, reviewing datasets across eight categories: semantics, object pose estimation, camera tracking, scene reconstruction, object tracking, human actions, faces and identification. By extracting relevant information in each category we help researchers to find appropriate data for their needs, and we consider which datasets have succeeded in driving computer vision forward and why. Finally, we examine the future of RGBD datasets. We identify key areas which are currently underexplored, and suggest that future directions may include synthetic data and dense reconstructions of static and dynamic scenes.Comment: 8 pages excluding references (CVPR style

Use of nonintrusive sensor-based information and communication technology for real-world evidence for clinical trials in dementia

Cognitive function is an important end point of treatments in dementia clinical trials. Measuring cognitive function by standardized tests, however, is biased toward highly constrained environments (such as hospitals) in selected samples. Patient-powered real-world evidence using information and communication technology devices, including environmental and wearable sensors, may help to overcome these limitations. This position paper describes current and novel information and communication technology devices and algorithms to monitor behavior and function in people with prodromal and manifest stages of dementia continuously, and discusses clinical, technological, ethical, regulatory, and user-centered requirements for collecting real-world evidence in future randomized controlled trials. Challenges of data safety, quality, and privacy and regulatory requirements need to be addressed by future smart sensor technologies. When these requirements are satisfied, these technologies will provide access to truly user relevant outcomes and broader cohorts of participants than currently sampled in clinical trials

The detection of concealed firearm carrying trough CCTV: the role of affect recognition

This research aimed to explore whether the recognition of offenders with a concealed firearm by a human operator might be based on the recognition of affective (negative) state derived from non-verbal behaviour that is accessible from CCTV images. Since a firearm is concealed, it has been assumed that human observers would respond to subtle cues which individuals inherently produce whilst carrying a hidden firearm. These cues are believed to be reflected in the body language of those carrying firearms and might be apprehended by observers at a conscious or subconscious level. Another hypothesis is that the ability to recognize the carrier of concealed firearm in the CCTV footage might be affected by other factors, such as the skills in decoding an affective state of others and the viewpoint of observation of the surveillance targets. In order to give a theoretical and experimental basis for these hypotheses the first objective was to examine the extant literature to determine what is known about recognition of affect from non-verbal cues (e.g. facial expressions and body movement), and how it can be applied to the detection of human mal-intent. A second objective was to explore this subject in relation to the detection of concealed firearm carrying through performing a number of experimental studies. The studies employed experts, i.e. CCTV operators and mainly the lay people as participants. Also, various experimental techniques such as questionnaires and eye-tracking registration were used to investigate the topic. The results show that human observers seem to use visual indicators of affective state of surveillance targets to make a decision whether or not the individuals are carrying a concealed firearm. The most prominent cues were face, and upper body of surveillance targets, gait, posture and arm movements. The test of decoding ability did not show sufficient relationship with the ability to detect a concealed firearm bearer. The performance on the task might be view dependent. Further research into this topic will be needed to generate strategies that would support reliable detection of concealed firearm carrying through employing of related affective behavioural cues