Facial Video-based Remote Physiological Measurement via Self-supervised Learning

Facial video-based remote physiological measurement aims to estimate remote photoplethysmography (rPPG) signals from human face videos and then measure multiple vital signs (e.g. heart rate, respiration frequency) from rPPG signals. Recent approaches achieve it by training deep neural networks, which normally require abundant facial videos and synchronously recorded photoplethysmography (PPG) signals for supervision. However, the collection of these annotated corpora is not easy in practice. In this paper, we introduce a novel frequency-inspired self-supervised framework that learns to estimate rPPG signals from facial videos without the need of ground truth PPG signals. Given a video sample, we first augment it into multiple positive/negative samples which contain similar/dissimilar signal frequencies to the original one. Specifically, positive samples are generated using spatial augmentation. Negative samples are generated via a learnable frequency augmentation module, which performs non-linear signal frequency transformation on the input without excessively changing its visual appearance. Next, we introduce a local rPPG expert aggregation module to estimate rPPG signals from augmented samples. It encodes complementary pulsation information from different face regions and aggregate them into one rPPG prediction. Finally, we propose a series of frequency-inspired losses, i.e. frequency contrastive loss, frequency ratio consistency loss, and cross-video frequency agreement loss, for the optimization of estimated rPPG signals from multiple augmented video samples and across temporally neighboring video samples. We conduct rPPG-based heart rate, heart rate variability and respiration frequency estimation on four standard benchmarks. The experimental results demonstrate that our method improves the state of the art by a large margin.Comment: IEEE Transactions on Pattern Analysis and Machine Intelligenc

Enhancing Space-time Video Super-resolution via Spatial-temporal Feature Interaction

The target of space-time video super-resolution (STVSR) is to increase both the frame rate (also referred to as the temporal resolution) and the spatial resolution of a given video. Recent approaches solve STVSR with end-to-end deep neural networks. A popular solution is to first increase the frame rate of the video; then perform feature refinement among different frame features; and last increase the spatial resolutions of these features. The temporal correlation among features of different frames is carefully exploited in this process. The spatial correlation among features of different (spatial) resolutions, despite being also very important, is however not emphasized. In this paper, we propose a spatial-temporal feature interaction network to enhance STVSR by exploiting both spatial and temporal correlations among features of different frames and spatial resolutions. Specifically, the spatial-temporal frame interpolation module is introduced to interpolate low- and high-resolution intermediate frame features simultaneously and interactively. The spatial-temporal local and global refinement modules are respectively deployed afterwards to exploit the spatial-temporal correlation among different features for their refinement. Finally, a novel motion consistency loss is employed to enhance the motion continuity among reconstructed frames. We conduct experiments on three standard benchmarks, Vid4, Vimeo-90K and Adobe240, and the results demonstrate that our method improves the state of the art methods by a considerable margin. Our codes will be available at https://github.com/yuezijie/STINet-Space-time-Video-Super-resolution

A Novel Non-Volatile Inverter-based CiM: Continuous Sign Weight Transition and Low Power on-Chip Training

In this work, we report a novel design, one-transistor-one-inverter (1T1I), to satisfy high speed and low power on-chip training requirements. By leveraging doped HfO2 with ferroelectricity, a non-volatile inverter is successfully demonstrated, enabling desired continuous weight transition between negative and positive via the programmable threshold voltage (VTH) of ferroelectric field-effect transistors (FeFETs). Compared with commonly used designs with the similar function, 1T1I uniquely achieves pure on-chip-based weight transition at an optimized working current without relying on assistance from off-chip calculation units for signed-weight comparison, facilitating high-speed training at low power consumption. Further improvements in linearity and training speed can be obtained via a two-transistor-one-inverter (2T1I) design. Overall, focusing on energy and time efficiencies, this work provides a valuable design strategy for future FeFET-based computing-in-memory (CiM)

A promoting role of androgen receptor in androgen-sensitive and -insensitive prostate cancer cells

Although the vital role of the androgen receptor (AR) has been well demonstrated in primary prostate cancers, its role in the androgen-insensitive prostate cancers still remains unclear. Here, we used a small hairpin RNA approach to directly assess AR activity in prostate cancer cells. Reduction of AR expression in the two androgen-sensitive prostate cancer cell lines, LNCaP and LAPC4, significantly decreased AR-mediated transcription and cell growth. Intriguingly, in two androgen-insensitive prostate cell lines, LNCaP-C42B4 and CWR22Rv1, knockdown of AR expression showed a more pronounced effect on AR-induced transcription and cell growth than androgen depletion. Using cDNA microarrays, we also compared the transcriptional profiles induced by either androgen depletion or AR knockdown. Although a significant number of transcripts appear to be regulated by both androgen depletion and AR knockdown, we observed a subset of transcripts affected only by androgen depletion but not by AR knockdown, and vice versa. Finally, we demonstrated a direct role for AR in promoting tumor formation and growth in a xenograft model. Taken together, our results elucidate an important role for the AR in androgen-insensitive prostate cancer cells, and suggest that AR can be used as a therapeutic target for androgen-insensitive prostate cancers

Single-cell chromatin accessibility profiling of cell-state-specific gene regulatory programs during mouse organogenesis

In mammals, early organogenesis begins soon after gastrulation, accompanied by specification of various type of progenitor/precusor cells. In order to reveal dynamic chromatin landscape of precursor cells and decipher the underlying molecular mechanism driving early mouse organogenesis, we performed single-cell ATAC-seq of E8.5-E10.5 mouse embryos. We profiled a total of 101,599 single cells and identified 41 specific cell types at these stages. Besides, by performing integrated analysis of scATAC-seq and public scRNA-seq data, we identified the critical cis-regulatory elements and key transcription factors which drving development of spinal cord and somitogenesis. Furthermore, we intersected accessible peaks with human diseases/traits-related loci and found potential clinical associated single nucleotide variants (SNPs). Overall, our work provides a fundamental source for understanding cell fate determination and revealing the underlying mechanism during postimplantation embryonic development, and expand our knowledge of pathology for human developmental malformations

Wheat Teacher: A One-Stage Anchor-Based Semi-Supervised Wheat Head Detector Utilizing Pseudo-Labeling and Consistency Regularization Methods

Wheat breeding heavily relies on the observation of various traits during the wheat growth process. Among all traits, wheat head density stands out as a particularly crucial characteristic. Despite the realization of high-throughput phenotypic data collection for wheat, the development of efficient and robust models for extracting traits from raw data remains a significant challenge. Numerous fully supervised target detection algorithms have been employed to address the wheat head detection problem. However, constrained by the exorbitant cost of dataset creation, especially the manual annotation cost, fully supervised target detection algorithms struggle to unleash their full potential. Semi-supervised training methods can leverage unlabeled data to enhance model performance, addressing the issue of insufficient labeled data. This paper introduces a one-stage anchor-based semi-supervised wheat head detector, named “Wheat Teacher”, which combines two semi-supervised methods, pseudo-labeling, and consistency regularization. Furthermore, two novel dynamic threshold components, Pseudo-label Dynamic Allocator and Loss Dynamic Threshold, are designed specifically for wheat head detection scenarios to allocate pseudo-labels and filter losses. We conducted detailed experiments on the largest wheat head public dataset, GWHD2021. Compared with various types of detectors, Wheat Teacher achieved a mAP0.5 of 92.8% with only 20% labeled data. This result surpassed the test outcomes of two fully supervised object detection models trained with 100% labeled data, and the difference with the other two fully supervised models trained with 100% labeled data was within 1%. Moreover, Wheat Teacher exhibits improvements of 2.1%, 3.6%, 5.1%, 37.7%, and 25.8% in mAP0.5 under different labeled data usage ratios of 20%, 10%, 5%, 2%, and 1%, respectively, validating the effectiveness of our semi-supervised approach. These experiments demonstrate the significant potential of Wheat Teacher in wheat head detection