Multi-Modality Human Action Recognition

Human action recognition is very useful in many applications in various areas, e.g. video surveillance, HCI (Human computer interaction), video retrieval, gaming and security. Recently, human action recognition becomes an active research topic in computer vision and pattern recognition. A number of action recognition approaches have been proposed. However, most of the approaches are designed on the RGB images sequences, where the action data was collected by RGB/intensity camera. Thus the recognition performance is usually related to various occlusion, background, and lighting conditions of the image sequences. If more information can be provided along with the image sequences, more data sources other than the RGB video can be utilized, human actions could be better represented and recognized by the designed computer vision system.;In this dissertation, the multi-modality human action recognition is studied. On one hand, we introduce the study of multi-spectral action recognition, which involves the information from different spectrum beyond visible, e.g. infrared and near infrared. Action recognition in individual spectra is explored and new methods are proposed. Then the cross-spectral action recognition is also investigated and novel approaches are proposed in our work. On the other hand, since the depth imaging technology has made a significant progress recently, where depth information can be captured simultaneously with the RGB videos. The depth-based human action recognition is also investigated. I first propose a method combining different type of depth data to recognize human actions. Then a thorough evaluation is conducted on spatiotemporal interest point (STIP) based features for depth-based action recognition. Finally, I advocate the study of fusing different features for depth-based action analysis. Moreover, human depression recognition is studied by combining facial appearance model as well as facial dynamic model

AnomalyHop : An SSL-based Image Anomaly Localization Method

An image anomaly localization method based on the successive subspace learning (SSL) framework, called Anomaly-Hop, is proposed in this work. AnomalyHop consists of three modules: 1) feature extraction via successive subspace learning (SSL), 2) normality feature distributions modeling via Gaussian models, and 3) anomaly map generation and fusion. Comparing with state-of-the-art image anomaly localization methods based on deep neural networks (DNNs), AnomalyHop is mathematically transparent, easy to train, and fast in its inference speed. Besides, its area under the ROC curve (ROC-AUC) performance on the MVTec AD dataset is 95.9%, which is among the best of several benchmarking methods.acceptedVersionPeer reviewe

Learning task-specific similarity

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, February 2006.Includes bibliographical references (p. 139-147).The right measure of similarity between examples is important in many areas of computer science. In particular it is a critical component in example-based learning methods. Similarity is commonly defined in terms of a conventional distance function, but such a definition does not necessarily capture the inherent meaning of similarity, which tends to depend on the underlying task. We develop an algorithmic approach to learning similarity from examples of what objects are deemed similar according to the task-specific notion of similarity at hand, as well as optional negative examples. Our learning algorithm constructs, in a greedy fashion, an encoding of the data. This encoding can be seen as an embedding into a space, where a weighted Hamming distance is correlated with the unknown similarity. This allows us to predict when two previously unseen examples are similar and, importantly, to efficiently search a very large database for examples similar to a query. This approach is tested on a set of standard machine learning benchmark problems. The model of similarity learned with our algorithm provides and improvement over standard example-based classification and regression. We also apply this framework to problems in computer vision: articulated pose estimation of humans from single images, articulated tracking in video, and matching image regions subject to generic visual similarity.by Gregory Shakhnarovich.Ph.D

A longitudinal study of the experiences and psychological well-being of Indian surrogates

Study question: What is the psychological well-being of Indian surrogates during and after the surrogacy pregnancy? Summary answer: Surrogates were similar to a matched group of expectant mothers on anxiety and stress. However, they scored higher on depression during and after pregnancy. What is known already: The recent ban on trans-national commercial surrogacy in India has led to urgent policy discussions regarding surrogacy. Whilst previous studies have reported the motivations and experiences of Indian surrogates no studies have systematically examined the psychological well-being of Indian surrogates, especially from a longitudinal perspective. Previous research has shown that Indian surrogates are motivated by financial payment and may face criticism from their family and community due to negative social stigma attached to surrogacy. Indian surrogates often recruited by agencies and mainly live together in a “surrogacy house.” Study design, size, duration: A longitudinal study was conducted comparing surrogates to a matched group of expectant mothers over two time points: (a) during pregnancy (Phase1: 50 surrogates, 70 expectant mothers) and (b) 4–6 months after delivery (Phase 2: 45 surrogates, 49 expectant mothers). The Surrogates were recruited from a fertility clinic in Mumbai and the matched comparison group was recruited from four public hospitals in Mumbai and Delhi. Data collection was completed over 2 years. Participants/materials, setting, methods: Surrogates and expectant mothers were aged between 23 and 36 years. All participants were from a low socio-economic background and had left school before 12–13 years of age. In-depth faceto-face semi-structured interviews and a psychological questionnaire assessing anxiety, stress and depression were administered in Hindi to both groups. Interviews took place in a private setting. Audio recordings of surrogate interviews were later translated and transcribed into English. Main results and the role of chance: Stress and anxiety levels did not significantly differ between the two groups for both phases of the study. For depression, surrogates were found to be significantly more depressed than expectant mothers at phase 1 (p = 0.012) and phase 2 (p = 0.017). Within the surrogacy group, stress and depression did not change during and after pregnancy. However, a non-significant trend was found showing that anxiety decreased after delivery (p = 0.086). No participants reported being coerced into surrogacy, however nearly all kept it a secret from their wider family and community and hence did not face criticism. Surrogates lived at the surrogate house for different durations. During pregnancy, 66% (N = 33/50) reported their experiences of the surrogate house as positive, 24% (N = 12/50) as negative and 10% (N = 5/50) as neutral. After delivery, most surrogates (66%, N = 30/45) reported their experiences of surrogacy to be positive, with the remainder viewing it as neutral (28%) or negative (4%). In addition, most (66%, N = 30/45) reported that they had felt “socially supported and loved” during the surrogacy arrangement by friends in the surrogate hostel, clinic staff or family. Most surrogates did not meet the intending parents (49%, N = 22/45) or the resultant child (75%, N = 34/45). Limitations, reasons for caution: Since the surrogates were recruited from only one clinic, the findings may not be representative of all Indian surrogates. Some were lost to follow-up which may have produced sampling bias. Wider implications of the findings: This is the first study to examine the psychological well-being of surrogates in India. This research is of relevance to current policy discussions in India regarding legislation on surrogacy. Moreover, the findings are of relevance to clinicians, counselors and other professionals involved in surrogacy. Trial registration number: N/A