Center for Digital Health (CDH)

As part of the mini-symposium entitled “Research on Digital Health for Designing Scalable Pervasive Healthcare Monitoring, Rehabilitation, and Home-based Healthcare Systems,” Dr. Ma discusses the UMass Center for Digital Health, for which he is Co-Director

Facial Action Unit Detection Using Attention and Relation Learning

Attention mechanism has recently attracted increasing attentions in the field of facial action unit (AU) detection. By finding the region of interest of each AU with the attention mechanism, AU-related local features can be captured. Most of the existing attention based AU detection works use prior knowledge to predefine fixed attentions or refine the predefined attentions within a small range, which limits their capacity to model various AUs. In this paper, we propose an end-to-end deep learning based attention and relation learning framework for AU detection with only AU labels, which has not been explored before. In particular, multi-scale features shared by each AU are learned firstly, and then both channel-wise and spatial attentions are adaptively learned to select and extract AU-related local features. Moreover, pixel-level relations for AUs are further captured to refine spatial attentions so as to extract more relevant local features. Without changing the network architecture, our framework can be easily extended for AU intensity estimation. Extensive experiments show that our framework (i) soundly outperforms the state-of-the-art methods for both AU detection and AU intensity estimation on the challenging BP4D, DISFA, FERA 2015 and BP4D+ benchmarks, (ii) can adaptively capture the correlated regions of each AU, and (iii) also works well under severe occlusions and large poses.Comment: This paper is accepted by IEEE Transactions on Affective Computin

Mitigating Transformer Overconfidence via Lipschitz Regularization

Though Transformers have achieved promising results in many computer vision tasks, they tend to be over-confident in predictions, as the standard Dot Product Self-Attention (DPSA) can barely preserve distance for the unbounded input domain. In this work, we fill this gap by proposing a novel Lipschitz Regularized Transformer (LRFormer). Specifically, we present a new similarity function with the distance within Banach Space to ensure the Lipschitzness and also regularize the term by a contractive Lipschitz Bound. The proposed method is analyzed with a theoretical guarantee, providing a rigorous basis for its effectiveness and reliability. Extensive experiments conducted on standard vision benchmarks demonstrate that our method outperforms the state-of-the-art single forward pass approaches in prediction, calibration, and uncertainty estimation.Comment: Accepted by UAI 2023. (https://proceedings.mlr.press/v216/ye23a.html

Radar Enlighten the Dark: Enhancing Low-Visibility Perception for Automated Vehicles with Camera-Radar Fusion

Sensor fusion is a crucial augmentation technique for improving the accuracy and reliability of perception systems for automated vehicles under diverse driving conditions. However, adverse weather and low-light conditions remain challenging, where sensor performance degrades significantly, exposing vehicle safety to potential risks. Advanced sensors such as LiDARs can help mitigate the issue but with extremely high marginal costs. In this paper, we propose a novel transformer-based 3D object detection model "REDFormer" to tackle low visibility conditions, exploiting the power of a more practical and cost-effective solution by leveraging bird's-eye-view camera-radar fusion. Using the nuScenes dataset with multi-radar point clouds, weather information, and time-of-day data, our model outperforms state-of-the-art (SOTA) models on classification and detection accuracy. Finally, we provide extensive ablation studies of each model component on their contributions to address the above-mentioned challenges. Particularly, it is shown in the experiments that our model achieves a significant performance improvement over the baseline model in low-visibility scenarios, specifically exhibiting a 31.31% increase in rainy scenes and a 46.99% enhancement in nighttime scenes.The source code of this study is publicly available

The Association between Breakfast Skipping and Body Weight, Nutrient Intake, and Metabolic Measures among Participants with Metabolic Syndrome

The effect of skipping breakfast on health, especially in adults, remains a controversial topic. A secondary data analysis was conducted to examine associations between breakfast eating patterns and weight loss, nutrient intake, and metabolic parameters among participants with metabolic syndrome (MetS) (n = 240). Three randomly selected 24-h dietary recalls were collected from each participant at baseline and at the one-year visit. Skipped breakfast was seen in 32.9% at baseline and in 17.4% at the one-year visit, respectively. At baseline, after adjustment for demographics and physical activity, participants who ate breakfast had a higher thiamin, niacin, and folate intake than did breakfast skippers (p \u3c 0.05); other selected parameters including body weight, dietary quality scores, nutrient intake, and metabolic parameters showed no significant differences between the two groups (p ≥ 0.05). From baseline to one year, after adjustment for covariates, mean fat intake increased by 2.7% (95% confidence intervals (CI): −1.0, 6.5%) of total energy in breakfast skippers in comparison to the 1.2% decrease observed in breakfast eaters (95% CI: −3.4, 1.1%) (p = 0.02). Mean changes in other selected parameters showed no significant differences between breakfast skippers and eaters (p \u3e 0.05). This study did not support the hypothesis that skipping breakfast has impact on body weight, nutrient intakes, and selected metabolic measures in participants with MetS

Spatio-Temporal Relation and Attention Learning for Facial Action Unit Detection

Spatio-temporal relations among facial action units (AUs) convey significant information for AU detection yet have not been thoroughly exploited. The main reasons are the limited capability of current AU detection works in simultaneously learning spatial and temporal relations, and the lack of precise localization information for AU feature learning. To tackle these limitations, we propose a novel spatio-temporal relation and attention learning framework for AU detection. Specifically, we introduce a spatio-temporal graph convolutional network to capture both spatial and temporal relations from dynamic AUs, in which the AU relations are formulated as a spatio-temporal graph with adaptively learned instead of predefined edge weights. Moreover, the learning of spatio-temporal relations among AUs requires individual AU features. Considering the dynamism and shape irregularity of AUs, we propose an attention regularization method to adaptively learn regional attentions that capture highly relevant regions and suppress irrelevant regions so as to extract a complete feature for each AU. Extensive experiments show that our approach achieves substantial improvements over the state-of-the-art AU detection methods on BP4D and especially DISFA benchmarks

Normalized, HOS-Based, Blind Speech Separation Algorithms

Abstrac

Methodology of an exercise intervention program using social incentives and gamification for obese children

BACKGROUND: Traditional exercise [supervised exercise (SE)] intervention has been proved to be one of the most effective ways to improve metabolic health. However, most exercise interventions were on a high-cost and small scale, moreover lacking of the long-term effect due to low engagement. On the other hand, it was noteworthy that gamification and social incentives were promising strategies to increase engagement and sustain exercise interventions effects; as well as mobile technologies such as WeChat also can provide an appropriate platform to deploy interventions on a broader, low-cost scale. Thus, we aim to develop a novel exercise intervention (\u27SandG exercise intervention\u27) that combines SE intervention with gamification and social incentives design through WeChat, with the aim of improving metabolic health and poor behaviors among overweight and obesity children. METHODS: We propose a randomized controlled trial of a \u27SandG exercise intervention\u27 among 420 overweight and obese children who have at least one marker of metabolic syndrome. Children will be randomized to control or intervention group in a 1:1 ratio. The exercise intervention package includes intervention designs based on integrated social incentives and gamification theory, involving targeted essential volume and intensity of activity (skipping rope) as well as monitoring daily information and providing health advice by WeChat. Participants will undertake assessments at baseline, at end of intervention period, in the follow-up time at months 3,6,12. The primary outcome is outcome of metabolic health. Secondary outcomes include behavioral (e.g., diary physical activity, diet) and anthropometric measures (e.g., body fat rate and muscle mass). DISCUSSIONS: This will be the first study to design an exercise intervention model that combines traditional supervised exercise (SE) intervention with gamification and social incentives theory through WeChat. We believed that this study could explore a low-cost, easy-to-popularize, and effective exercise intervention model for improving metabolic health and promote healthy among obese children. Furthermore, it will also provide important evidence for guidelines to prevent and improve metabolic health and health behaviors. TRIAL REGISTRATION: 10-04-2019;Registration number: ChiCTR1900022396

Pelvic prehabilitation: pelvic exercises assist in minimizing inter-fraction sacral slope variability during radiation therapy

Introduction: Prehabilitation for radiation therapy is not well studied. Retrospective data shows variability in set-up positioning of patients during daily pelvic RT. We hypothesize that a brief structured daily exercise regimen is feasible for subjects to perform before RT and may minimize variability in positioning as measured by sacral slope angles (SSA) on lateral views. Determining feasibility and effectiveness of these exercises in decreasing set-up variability has clinical implications, both for targeting treatment sites and preventing adverse effects. Methods: Subjects in the exercise intervention condition (n=8, 8 F) performed a structured daily hip exercise regimen throughout the duration of RT, and subjects in the historical control condition (n=20, 17 F, 3 M) had usual care. For each patient, SSA measurements were compared to SSA measurements from the simulation CT for 5 weeks during RT. The extent of variability of measurements between two conditions was studied using a linear mixed model. For all patients in both conditions, the same two readers independently measured SSA to compare angles on day of simulation against the angles measured from each day of RT. Results: The average variation in SSA for intervention condition was 0.913° (±0.582°), with range among patients 0.57°-1.3°. The average variation for control condition was 2.27° (±1.43°), with range among patients 1.22° - 5.09°. The difference between two conditions was statistically significant (p=0.0019). Comparison of SSA variation between conditions demonstrated a statistically significant difference at each week (wk 1: p = 0.0071, wk 2: p = 0.0077, wk 3: p = 0.011, wk 4: p = 0.005, wk 5: p = 0.0079). The exercise intervention condition had no significant variation between week 1 and later weeks (wk 2: p = 0.876, wk 3: p = 0.741, wk 4: p = 0.971, wk 5: p = 0.397). The control condition showed greater SSA variation between week 1 and later weeks (wk 2: p = 0.868, wk 3: p = 0.915, wk 4: p = 0.015, wk 5: p = 0.224), with significant variation between weeks 1 and 4. No subject reported any adverse effects. Conclusion: We observed a significant decrease in sacral slope variability in our exercise cohort as compared to historical controls. SSA variation for control condition increased over the course of treatment with significant difference noted between week 1 and 4. A larger clinical trial is required to evaluate the potential clinical benefits of a structured daily exercise regimen during pelvic RT. References: Silver JK, Baima J. Cancer prehabilitation: an opportunity to decrease treatment-related morbidity, increase cancer treatment options, and improve physical and psychological health outcomes. American journal of physical medicine & rehabilitation. 2013 Aug 1;92(8):715-27. Lukez A, O’Loughlin L, Bodla M, Baima J, Moni J. Positioning of port films for radiation: variability is present. Medical Oncology. 2018 May 1;35(5):77. Kwon JW, Huh SJ, Yoon YC, Choi SH, Jung JY, Oh D, Choe BK. Pelvic bone complications after radiation therapy of uterine cervical cancer: evaluation with MRI. American Journal of Roentgenology. 2008 Oct;191(4):987-94. Stubblefield MD. Radiation fibrosis syndrome: neuromuscular and musculoskeletal complications in cancer survivors. PM&R. 2011 Nov 1;3(11):1041-54

Bayesian variable selection for high dimensional predictors and self-reported outcomes

BACKGROUND: The onset of silent diseases such as type 2 diabetes is often registered through self-report in large prospective cohorts. Self-reported outcomes are cost-effective; however, they are subject to error. Diagnosis of silent events may also occur through the use of imperfect laboratory-based diagnostic tests. In this paper, we describe an approach for variable selection in high dimensional datasets for settings in which the outcome is observed with error. METHODS: We adapt the spike and slab Bayesian Variable Selection approach in the context of error-prone, self-reported outcomes. The performance of the proposed approach is studied through simulation studies. An illustrative application is included using data from the Women\u27s Health Initiative SNP Health Association Resource, which includes extensive genotypic ( \u3e 900,000 SNPs) and phenotypic data on 9,873 African American and Hispanic American women. RESULTS: Simulation studies show improved sensitivity of our proposed method when compared to a naive approach that ignores error in the self-reported outcomes. Application of the proposed method resulted in discovery of several single nucleotide polymorphisms (SNPs) that are associated with risk of type 2 diabetes in a dataset of 9,873 African American and Hispanic participants in the Women\u27s Health Initiative. There was little overlap among the top ranking SNPs associated with type 2 diabetes risk between the racial groups, adding support to previous observations in the literature of disease associated genetic loci that are often not generalizable across race/ethnicity populations. The adapted Bayesian variable selection algorithm is implemented in R. The source code for the simulations are available in the Supplement. CONCLUSIONS: Variable selection accuracy is reduced when the outcome is ascertained by error-prone self-reports. For this setting, our proposed algorithm has improved variable selection performance when compared to approaches that neglect to account for the error-prone nature of self-reports