Towards Reading Hidden Emotions: A comparative Study of Spontaneous Micro-expression Spotting and Recognition Methods

Micro-expressions (MEs) are rapid, involuntary facial expressions which reveal emotions that people do not intend to show. Studying MEs is valuable as recognizing them has many important applications, particularly in forensic science and psychotherapy. However, analyzing spontaneous MEs is very challenging due to their short duration and low intensity. Automatic ME analysis includes two tasks: ME spotting and ME recognition. For ME spotting, previous studies have focused on posed rather than spontaneous videos. For ME recognition, the performance of previous studies is low. To address these challenges, we make the following contributions: (i)We propose the first method for spotting spontaneous MEs in long videos (by exploiting feature difference contrast). This method is training free and works on arbitrary unseen videos. (ii)We present an advanced ME recognition framework, which outperforms previous work by a large margin on two challenging spontaneous ME databases (SMIC and CASMEII). (iii)We propose the first automatic ME analysis system (MESR), which can spot and recognize MEs from spontaneous video data. Finally, we show our method outperforms humans in the ME recognition task by a large margin, and achieves comparable performance to humans at the very challenging task of spotting and then recognizing spontaneous MEs

Stresses Within the Actin Meshwork Control the Turnover of Fimbrin During Clathrin-Mediated Endocytosis

In this dissertation, I investigated the molecular mechanism of clathrin-mediated endocytosis (CME) in fission yeast with a sparse labeling strategy to track endocytic proteins at the single molecule level. CME is involved in a variety of biological processes, such as nutrient internalization and receptor recycling. CME is also a well-conserved biological process from yeast to mammalian cells. During clathrin-mediated endocytosis, about 60 different endocytic proteins are recruited to the endocytic site in a highly reproducible order. During the endocytic event, endocytic proteins assemble into endocytic structures, contributing to membrane invagination and endocytic vesicle formation. Based on the single molecule endocytic protein trajectories I obtained, I proved the significance of stresses within the actin meshwork. I also investigated the dwell-time distribution of single molecules of fimbrin (a protein that crosslinks actin filaments) and provide new mechanisms for fimbrin-actin binding mechanism. To study the single-molecule endocytic protein dynamics, I upgraded a two-color Total Internal Reflection Fluorescence (TIRF) microscopy system to study the single molecule dynamics of endocytic proteins. The two-color imaging system can be applied to probe relative motions between endocytic proteins in further studies

Feature Selection with Cost Constraint

When acquiring consumer data for marketing or new business initiatives, it is important to decide what features of potential customers should be acquired. We study feature selection and acquisition problem with cost constraint in the context of regression prediction. We formulate the feature selection and acquisition problem as a nonlinear programming problem that minimizes prediction error and number of features used in the model subject to a budget constraint. We derive the analytical properties of the solution for this problem and provide a computational procedure for solving the problem. The results of a preliminary experiment demonstrate the effectiveness of our approach

Contrast-Phys: Unsupervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast

Video-based remote physiological measurement utilizes face videos to measure the blood volume change signal, which is also called remote photoplethysmography (rPPG). Supervised methods for rPPG measurements achieve state-of-the-art performance. However, supervised rPPG methods require face videos and ground truth physiological signals for model training. In this paper, we propose an unsupervised rPPG measurement method that does not require ground truth signals for training. We use a 3DCNN model to generate multiple rPPG signals from each video in different spatiotemporal locations and train the model with a contrastive loss where rPPG signals from the same video are pulled together while those from different videos are pushed away. We test on five public datasets, including RGB videos and NIR videos. The results show that our method outperforms the previous unsupervised baseline and achieves accuracies very close to the current best supervised rPPG methods on all five datasets. Furthermore, we also demonstrate that our approach can run at a much faster speed and is more robust to noises than the previous unsupervised baseline. Our code is available at https://github.com/zhaodongsun/contrast-phys.Comment: accepted to ECCV 202

Counterfactual Explanations for Incorrect Predictions Made by AI Models

Advanced AI models are powerful in making accurate predictions for complex problems. However, these models often operate as black boxes. This lack of interpretability poses significant challenges, especially in high-stakes applications such as finance, healthcare, and criminal justice. Explainable AI seeks to address the challenges by developing methods that can provide meaningful explanations for humans to understand. When black box models are used for prediction, they inevitably produce errors. It is important to appropriately explain incorrect predictions. This problem, however, has not been addressed in the literature. In this study, we propose a novel method to provide explanations for misclassified cases made by black box models. The proposed method takes a counterfactual explanation approach. It builds a decision tree to find the best counterfactual examples for explanations. Incorrect predictions are rectified using a trust score measure. We validate the proposed method in an evaluation study using real-world data

Acquiring Second-Party Transaction Data for Customer Analytics

The recent development in second-party data market enables an organization to acquire customer data, including the customers’ individual transaction records or online behavior data, from the data owner that originally collects the data directly from its customers. This paper concerns individual-level second-party data acquisition under a budget constraint. Specifically, we focus on the problem of how to determine a set of customers whose data add most values to the organization for customer analytics. We model customer purchase behaviors using a hierarchical Bayesian modeling approach. We propose a novel data selection method for organizations to acquire individual-level data such that the acquired data are most useful for customer analytics problems. We evaluate the proposed method in an experimental study using real-world data. The results of the experimental evaluation demonstrate the effectiveness of our approach

Protecting Privacy When Releasing Search Results from Medical Document Data

Health information technologies have greatly facilitated sharing of personal health data for secondary use, which is critical to medical and health research. However, there is a growing concern about privacy due to data sharing and publishing. Medical and health data typically contain unstructured text documents, such as clinical narratives, pathology reports, and discharge summaries. This study concerns privacy-preserving extraction, summary, and release of information from medical documents. Existing studies on privacy-preserving data mining and publishing focus mostly on structured data. We propose a novel approach to enable privacy-preserving extract, summarize, query and report patients’ demographic, health and medical information from medical documents. The extracted data is represented in a semi-structured, set-valued data format, which can be stored in a health information system for query and analysis. The privacy preserving mechanism is based on the cutting-edge idea of differential privacy, which offers rigorous privacy guarantee

Parameterizing Topic Models for Empirical Research

Machine learning techniques have been increasingly employed in business research to discover or extract new simple features from large and unstructured data. These machine learned features (MLFs) are then used as independent or explanatory variables in the main econometric models for empirical research. Despite this growing trend, there has been little research regarding the impact of using MLFs on statistical inference for empirical research. In this paper, we undertake parameter estimation issues related to the use of topics/features extracted by Latent Dirichlet Allocation, a popular machine learning technique for text mining. We propose a novel method to extract features that result in the minimum-variance estimation of the regression model parameters. This enables a better use of unstructured text data for econometric modeling in empirical research. The effectiveness of the proposed method is validated with an experimental evaluation study on real-world text data