Orientation-Aware Leg Movement Learning for Action-Driven Human Motion Prediction

The task of action-driven human motion prediction aims to forecast future human motion from the observed sequence while respecting the given action label. It requires modeling not only the stochasticity within human motion but the smooth yet realistic transition between multiple action labels. However, the fact that most of the datasets do not contain such transition data complicates this task. Existing work tackles this issue by learning a smoothness prior to simply promote smooth transitions, yet doing so can result in unnatural transitions especially when the history and predicted motions differ significantly in orientations. In this paper, we argue that valid human motion transitions should incorporate realistic leg movements to handle orientation changes, and cast it as an action-conditioned in-betweening (ACB) learning task to encourage transition naturalness. Because modeling all possible transitions is virtually unreasonable, our ACB is only performed on very few selected action classes with active gait motions, such as Walk or Run. Specifically, we follow a two-stage forecasting strategy by first employing the motion diffusion model to generate the target motion with a specified future action, and then producing the in-betweening to smoothly connect the observation and prediction to eventually address motion prediction. Our method is completely free from the labeled motion transition data during training. To show the robustness of our approach, we generalize our trained in-betweening learning model on one dataset to two unseen large-scale motion datasets to produce natural transitions. Extensive methods on three benchmark datasets demonstrate that our method yields the state-of-the-art performance in terms of visual quality, prediction accuracy, and action faithfulness

Learning to Predict Diverse Human Motions from a Single Image via Mixture Density Networks

Human motion prediction, which plays a key role in computer vision, generally requires a past motion sequence as input. However, in real applications, a complete and correct past motion sequence can be too expensive to achieve. In this paper, we propose a novel approach to predicting future human motions from a much weaker condition, i.e., a single image, with mixture density networks (MDN) modeling. Contrary to most existing deep human motion prediction approaches, the multimodal nature of MDN enables the generation of diverse future motion hypotheses, which well compensates for the strong stochastic ambiguity aggregated by the single input and human motion uncertainty. In designing the loss function, we further introduce the energy-based formulation to flexibly impose prior losses over the learnable parameters of MDN to maintain motion coherence as well as improve the prediction accuracy by customizing the energy functions. Our trained model directly takes an image as input and generates multiple plausible motions that satisfy the given condition. Extensive experiments on two standard benchmark datasets demonstrate the effectiveness of our method in terms of prediction diversity and accuracy

Multilevel Saliency-Guided Self-Supervised Learning for Image Anomaly Detection

Anomaly detection (AD) is a fundamental task in computer vision. It aims to identify incorrect image data patterns which deviate from the normal ones. Conventional methods generally address AD by preparing augmented negative samples to enforce self-supervised learning. However, these techniques typically do not consider semantics during augmentation, leading to the generation of unrealistic or invalid negative samples. Consequently, the feature extraction network can be hindered from embedding critical features. In this study, inspired by visual attention learning approaches, we propose CutSwap, which leverages saliency guidance to incorporate semantic cues for augmentation. Specifically, we first employ LayerCAM to extract multilevel image features as saliency maps and then perform clustering to obtain multiple centroids. To fully exploit saliency guidance, on each map, we select a pixel pair from the cluster with the highest centroid saliency to form a patch pair. Such a patch pair includes highly similar context information with dense semantic correlations. The resulting negative sample is created by swapping the locations of the patch pair. Compared to prior augmentation methods, CutSwap generates more subtle yet realistic negative samples to facilitate quality feature learning. Extensive experimental and ablative evaluations demonstrate that our method achieves state-of-the-art AD performance on two mainstream AD benchmark datasets

Combinatory optimization of chromosomal integrated mevalonate pathway for β-carotene production in Escherichia coli

Additional file 1: Table S1. Primers used in this work. Table S2. Modulating genes of mvaS-mvaA-mavD1 operon for improving β-carotene production. Table S3. Modulating genes of Hmg1-erg12 operon for improving β-carotene production. Table S4. Sequences of representative artificial regulatory parts. Table S5. Plasmids used in this work. Table S6. Escherichia coli strains used in this work. Table S7. Calculated strength of mvaS and Hmg1 RBS, RBS sequence and relative β-carotene yield of strains from Re-modulation libraries. Figure S1. Two-step recombination method for inserting Hmg1-erg12 operon in E. coli chromosome. Figure S2. Two-step recombination method for modulating gene expression in E. coli chromosome by different artificial regulatory parts

A Multi-In and Multi-Out Dendritic Neuron Model and its Optimization

Artificial neural networks (ANNs), inspired by the interconnection of real neurons, have achieved unprecedented success in various fields such as computer vision and natural language processing. Recently, a novel mathematical ANN model, known as the dendritic neuron model (DNM), has been proposed to address nonlinear problems by more accurately reflecting the structure of real neurons. However, the single-output design limits its capability to handle multi-output tasks, significantly lowering its applications. In this paper, we propose a novel multi-in and multi-out dendritic neuron model (MODN) to tackle multi-output tasks. Our core idea is to introduce a filtering matrix to the soma layer to adaptively select the desired dendrites to regress each output. Because such a matrix is designed to be learnable, MODN can explore the relationship between each dendrite and output to provide a better solution to downstream tasks. We also model a telodendron layer into MODN to simulate better the real neuron behavior. Importantly, MODN is a more general and unified framework that can be naturally specialized as the DNM by customizing the filtering matrix. To explore the optimization of MODN, we investigate both heuristic and gradient-based optimizers and introduce a 2-step training method for MODN. Extensive experimental results performed on 11 datasets on both binary and multi-class classification tasks demonstrate the effectiveness of MODN, with respect to accuracy, convergence, and generality

ActiveSelfHAR: Incorporating Self Training into Active Learning to Improve Cross-Subject Human Activity Recognition

Deep learning-based human activity recognition (HAR) methods have shown great promise in the applications of smart healthcare systems and wireless body sensor network (BSN). Despite their demonstrated performance in laboratory settings, the real-world implementation of such methods is still hindered by the cross-subject issue when adapting to new users. To solve this issue, we propose ActiveSelfHAR, a framework that combines active learning's benefit of sparsely acquiring data with actual labels and self- training's benefit of effectively utilizing unlabeled data to enable the deep model to adapt to the target domain, i.e., the new users. In this framework, the model trained in the last iteration or the source domain is first utilized to generate pseudo labels of the target-domain samples and construct a self-training set based on the confidence score. Second, we propose to use the spatio-temporal relationships among the samples in the non-self-training set to augment the core set selected by active learning. Finally, we combine the self-training set and the augmented core set to fine-tune the model. We demonstrate our method by comparing it with state-of-the-art methods on two IMU-based datasets and an EMG-based dataset. Our method presents similar HAR accuracies with the upper bound, i.e. fully supervised fine-tuning with less than 1\% labeled data of the target dataset and significantly improves data efficiency and time cost. Our work highlights the potential of implementing user-independent HAR methods into smart healthcare systems and BSN

S-diclofenac Protects against Doxorubicin-Induced Cardiomyopathy in Mice via Ameliorating Cardiac Gap Junction Remodeling

Hydrogen sulfide (H2S), as a novel gaseous mediator, plays important roles in mammalian cardiovascular tissues. In the present study, we investigated the cardioprotective effect of S-diclofenac (2-[(2,6-dichlorophenyl)amino] benzeneacetic acid 4-(3H-1,2,dithiol-3-thione-5-yl)phenyl ester), a novel H2S-releasing derivative of diclofenac, in a murine model of doxorubicin-induced cardiomyopathy. After a single dose injection of doxorubicin (15 mg/kg, i.p.), male C57BL/6J mice were given daily treatment of S-diclofenac (25 and 50 µmol/kg, i.p.), diclofenac (25 and 50 µmol/kg, i.p.), NaHS (50 µmol/kg, i.p.), or same volume of vehicle. The cardioprotective effect of S-diclofenac was observed after 14 days. It showed that S-diclofenac, but not diclofenac, dose-dependently inhibited the doxorubicin-induced downregulation of cardiac gap junction proteins (connexin 43 and connexin 45) and thus reversed the remodeling of gap junctions in hearts. It also dose-dependently suppressed doxorubicin-induced activation of JNK in hearts. Furthermore, S-diclofenac produced a dose-dependent anti-inflammatory and anti-oxidative effect in this model. As a result, S-diclofenac significantly attenuated doxorubicin-related cardiac injury and cardiac dysfunction, and improved the survival rate of mice with doxorubicin-induced cardiomyopathy. These effects of S-diclofenac were mimicked in large part by NaHS. Therefore, we propose that H2S released from S-diclofenac in vivo contributes to the protective effect in doxorubicin-induced cardiomyopathy. These data also provide evidence for a critical role of H2S in the pathogenesis of doxorubicin-induced cardiomyopathy