Adaptive Multimodal Fusion For Facial Action Units Recognition

Multimodal facial action units (AU) recognition aims to build models that are capable of processing, correlating, and integrating information from multiple modalities (i.e., 2D images from a visual sensor, 3D geometry from 3D imaging, and thermal images from an infrared sensor). Although the multimodal data can provide rich information, there are two challenges that have to be addressed when learning from multimodal data: 1) the model must capture the complex cross-modal interactions in order to utilize the additional and mutual information effectively; 2) the model must be robust enough in the circumstance of unexpected data corruptions during testing, in case of a certain modality missing or being noisy. In this paper, we propose a novel Adaptive Multimodal Fusion method (AMF) for AU detection, which learns to select the most relevant feature representations from different modalities by a re-sampling procedure conditioned on a feature scoring module. The feature scoring module is designed to allow for evaluating the quality of features learned from multiple modalities. As a result, AMF is able to adaptively select more discriminative features, thus increasing the robustness to missing or corrupted modalities. In addition, to alleviate the over-fitting problem and make the model generalize better on the testing data, a cut-switch multimodal data augmentation method is designed, by which a random block is cut and switched across multiple modalities. We have conducted a thorough investigation on two public multimodal AU datasets, BP4D and BP4D+, and the results demonstrate the effectiveness of the proposed method. Ablation studies on various circumstances also show that our method remains robust to missing or noisy modalities during tests

Quantitative multidimensional phenotypes improve genetic analysis of laterality traits

The UK Medical Research Council and Wellcome (Grant ref: 217065/Z/19/Z) and the University of Bristol provide core support for ALSPAC. This publication is the work of the authors and SP and JS will serve as guarantors for the analysis of the ALSPAC data presented in this paper. GWAS data were generated by Sample Logistics and Genotyping Facilities at Wellcome Sanger Institute and LabCorp (Laboratory Corporation of America) using support from 23andMe. Support to the genetic analysis was provided by the St Andrews Bioinformatics Unit funded by the Wellcome Trust [grant 105621/Z/14/Z]. The Hong Kong sample was funded through a Collaborative Research Fund from the Hong Kong Special Administrative Region Research Grants Council (CUHK8/CRF/13G, and C4054-17WF). JS is funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation, 418445085) and supported by the Wellcome Trust [Institutional Strategic Support fund, Grant number 204821/Z/16/Z]. SP is funded by the Royal Society (UF150663).Handedness is the most commonly investigated lateralised phenotype and is usually measured as a binary left/right category. Its links with psychiatric and neurodevelopmental disorders prompted studies aimed at understanding the underlying genetics, while other measures and side preferences have been less studied. We investigated the heritability of hand, as well as foot, and eye preference by assessing parental effects (n ≤ 5028 family trios) and SNP-based heritability (SNP-h2, n ≤ 5931 children) in the Avon Longitudinal Study of Parents and Children (ALSPAC). An independent twin cohort from Hong Kong (n = 358) was used to replicate results from structural equation modelling (SEM). Parental left-side preference increased the chance of an individual to be left-sided for the same trait, with stronger maternal than paternal effects for footedness. By regressing out the effects of sex, age, and ancestry, we transformed laterality categories into quantitative measures. The SNP-h2 for quantitative handedness and footedness was 0.21 and 0.23, respectively, which is higher than the SNP-h2 reported in larger genetic studies using binary handedness measures. The heritability of the quantitative measure of handedness increased (0.45) compared to a binary measure for writing hand (0.27) in the Hong Kong twins. Genomic and behavioural SEM identified a shared genetic factor contributing to handedness, footedness, and eyedness, but no independent effects on individual phenotypes. Our analysis demonstrates how quantitative multidimensional laterality phenotypes are better suited to capture the underlying genetics than binary traits.Publisher PDFPeer reviewe

Flame Extinguishing Intelligent System

An ideal fire protection would be a system that is able to locate precisely where the fire is and focus water on that area only. This system should be able to tell whether the fire still exists or not and respond accordingly. Much less water will be wasted and water damage from this system will be minimized, compared to the current fire sprinkler system. In our project, our goal is to create a system that will respond to fire and that will be able to extinguish it. FlexiSys is a prototype flame extinguishing intelligent system which will detect flames in an area and proceed to extinguish the flames. Our product will ultimately be mounted on the ceiling, similarly to a passive fire sprinkler. Furthermore, we will try to minimize the power and space requirement, such that our product can replace current passive sprinkler systems with ease

Graphic Processing Units (GPUs)-Based Haptic Simulator for Dental Implant Surgery

This paper presents a haptics-based training simulator for dental implant surgery. Most of the previously developed dental simulators are targeted for exploring and drilling purpose only. The penalty-based contact force models with spherical-shaped dental tools are often adopted for simplicity and computational efficiency. In contrast, our simulator is equipped with a more precise force model adapted from the Voxmap-PointShell (VPS) method to capture the essential features of the drilling procedure, with no limitations on drill shape. In addition, a real-time torque model is proposed to simulate the torque resistance in the implant insertion procedure, based on patient-specific tissue properties and implant geometry. To achieve better anatomical accuracy, our oral model is reconstructed from cone beam computed tomography (CBCT) images with a voxel-based method. To enhance the real-time response, the parallel computing power of GPUs is exploited through extra efforts in data structure design, algorithms parallelization, and graphic memory utilization. Results show that the developed system can produce appropriate force feedback at different tissue layers during pilot drilling and can create proper resistance torque responses during implant insertion

Dysregulated protocadherin-pathway activity as an intrinsic defect in induced pluripotent stem cell-derived cortical interneurons from subjects with schizophrenia.

We generated cortical interneurons (cINs) from induced pluripotent stem cells derived from 14 healthy controls and 14 subjects with schizophrenia. Both healthy control cINs and schizophrenia cINs were authentic, fired spontaneously, received functional excitatory inputs from host neurons, and induced GABA-mediated inhibition in host neurons in vivo. However, schizophrenia cINs had dysregulated expression of protocadherin genes, which lie within documented schizophrenia loci. Mice lacking protocadherin-α showed defective arborization and synaptic density of prefrontal cortex cINs and behavioral abnormalities. Schizophrenia cINs similarly showed defects in synaptic density and arborization that were reversed by inhibitors of protein kinase C, a downstream kinase in the protocadherin pathway. These findings reveal an intrinsic abnormality in schizophrenia cINs in the absence of any circuit-driven pathology. They also demonstrate the utility of homogenous and functional populations of a relevant neuronal subtype for probing pathogenesis mechanisms during development

The evolution of synaptic and cognitive capacity: insights from the nervous system transcriptome of Aplysia

© The Author(s), 2022. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Orvis, J., Albertin, C., Shrestha, P., Chen, S., Zheng, M., Rodriguez, C., Tallon, L., Mahurkar, A., Zimin, A., Kim, M., Liu, K., Kandel, E., Fraser, C., Sossin, W., & Abrams, T. The evolution of synaptic and cognitive capacity: insights from the nervous system transcriptome of Aplysia. Proceedings of the National Academy of Sciences of the United States of America, 119(28), (2022): e2122301119, https://doi.org/10.1073/pnas.2122301119.The gastropod mollusk Aplysia is an important model for cellular and molecular neurobiological studies, particularly for investigations of molecular mechanisms of learning and memory. We developed an optimized assembly pipeline to generate an improved Aplysia nervous system transcriptome. This improved transcriptome enabled us to explore the evolution of cognitive capacity at the molecular level. Were there evolutionary expansions of neuronal genes between this relatively simple gastropod Aplysia (20,000 neurons) and Octopus (500 million neurons), the invertebrate with the most elaborate neuronal circuitry and greatest behavioral complexity? Are the tremendous advances in cognitive power in vertebrates explained by expansion of the synaptic proteome that resulted from multiple rounds of whole genome duplication in this clade? Overall, the complement of genes linked to neuronal function is similar between Octopus and Aplysia. As expected, a number of synaptic scaffold proteins have more isoforms in humans than in Aplysia or Octopus. However, several scaffold families present in mollusks and other protostomes are absent in vertebrates, including the Fifes, Lev10s, SOLs, and a NETO family. Thus, whereas vertebrates have more scaffold isoforms from select families, invertebrates have additional scaffold protein families not found in vertebrates. This analysis provides insights into the evolution of the synaptic proteome. Both synaptic proteins and synaptic plasticity evolved gradually, yet the last deuterostome-protostome common ancestor already possessed an elaborate suite of genes associated with synaptic function, and critical for synaptic plasticity.This work was supported by NSF EAGER Award IOS-1255695 and NIH grant R01 MH 55880 grant to T.W.A.; by a Natural Sciences and Engineering Research Council of Canada Discovery grant and Canadian Institutes of Health Research project grant 340328 to W.S.; by funding from the HHMI to E.R.K.; and by a Hibbitt Early Career Fellowship to C.A. W.S. is James McGill Professor at McGill University

The mechanisms of Yu Ping Feng San in tracking the cisplatin-resistance by regulating ATP-binding cassette transporter and glutathione S-transferase in lung cancer cells

Cisplatin is one of the first line anti-cancer drugs prescribed for treatment of solid tumors; however, the chemotherapeutic drug resistance is still a major obstacle of cisplatin in treating cancers. Yu Ping Feng San (YPFS), a well-known ancient Chinese herbal combination formula consisting of Astragali Radix, Atractylodis Macrocephalae Rhizoma and Saposhnikoviae Radix, is prescribed as a herbal decoction to treat immune disorders in clinic. To understand the fast-onset action of YPFS as an anti-cancer drug to fight against the drug resistance of cisplatin, we provided detailed analyses of intracellular cisplatin accumulation, cell viability, and expressions and activities of ATP-binding cassette transporters and glutathione S-transferases (GSTs) in YPFS-treated lung cancer cell lines. In cultured A549 or its cisplatin-resistance A549/DDP cells, application of YPFS increased accumulation of intracellular cisplatin, resulting in lower cell viability. In parallel, the activities and expressions of ATP-binding cassette transporters and GSTs were down-regulated in the presence of YPFS. The expression of p65 subunit of NF-κB complex was reduced by treating the cultures with YPFS, leading to a high ratio of Bax/Bcl-2, i.e. increasing the rate of cell death. Prim-O-glucosylcimifugin, one of the abundant ingredients in YPFS, modulated the activity of GSTs, and then elevated cisplatin accumulation, resulting in increased cell apoptosis. The present result supports the notion of YPFS in reversing drug resistance of cisplatin in lung cancer cells by elevating of intracellular cisplatin, and the underlying mechanism may be down regulating the activities and expressions of ATP-binding cassette transporters and GSTs

Wild Type and Mutant 2009 Pandemic Influenza A (H1N1) Viruses Cause More Severe Disease and Higher Mortality in Pregnant BALB/c Mice

BACKGROUND: Pregnant women infected by the pandemic influenza A (H1N1) 2009 virus had more severe disease and higher mortality but its pathogenesis is still unclear. PRINCIPAL FINDINGS: We showed that higher mortality, more severe pneumonitis, higher pulmonary viral load, lower peripheral blood T lymphocytes and antibody responses, higher levels of proinflammatory cytokines and chemokines, and worse fetal development occurred in pregnant mice than non-pregnant controls infected by either wild type (clinical isolate) or mouse-adapted mutant virus with D222G substitution in hemagglutinin. These disease-associated changes and the lower respiratory tract involvement were worse in pregnant mice challenged by mutant virus. Though human placental origin JEG-3 cell line could be infected and proinflammatory cytokines or chemokines were elevated in amniotic fluid of some mice, no placental or fetal involvement by virus were detected by culture, real-time reverse transcription polymerase chain reaction or histopathological changes. Dual immunofluorescent staining of viral nucleoprotein and type II alveolar cell marker SP-C protein suggested that the majority of infected alveolar epithelial cells were type II pneumocytes. CONCLUSION: The adverse effect of this pandemic virus on maternal and fetal outcome is largely related to the severe pulmonary disease and the indirect effect of inflammatory cytokine spillover into the systemic circulation

Identification of a Functional Genetic Variant at 16q12.1 for Breast Cancer Risk: Results from the Asia Breast Cancer Consortium

Genetic factors play an important role in the etiology of breast cancer. We carried out a multi-stage genome-wide association (GWA) study in over 28,000 cases and controls recruited from 12 studies conducted in Asian and European American women to identify genetic susceptibility loci for breast cancer. After analyzing 684,457 SNPs in 2,073 cases and 2,084 controls in Chinese women, we evaluated 53 SNPs for fast-track replication in an independent set of 4,425 cases and 1,915 controls of Chinese origin. Four replicated SNPs were further investigated in an independent set of 6,173 cases and 6,340 controls from seven other studies conducted in Asian women. SNP rs4784227 was consistently associated with breast cancer risk across all studies with adjusted odds ratios (95% confidence intervals) of 1.25 (1.20−1.31) per allele (P = 3.2×10−25) in the pooled analysis of samples from all Asian samples. This SNP was also associated with breast cancer risk among European Americans (per allele OR  = 1.19, 95% CI  = 1.09−1.31, P = 1.3×10−4, 2,797 cases and 2,662 controls). SNP rs4784227 is located at 16q12.1, a region identified previously for breast cancer risk among Europeans. The association of this SNP with breast cancer risk remained highly statistically significant in Asians after adjusting for previously-reported SNPs in this region. In vitro experiments using both luciferase reporter and electrophoretic mobility shift assays demonstrated functional significance of this SNP. These results provide strong evidence implicating rs4784227 as a functional causal variant for breast cancer in the locus 16q12.1 and demonstrate the utility of conducting genetic association studies in populations with different genetic architectures