Towards Good Practices for Missing Modality Robust Action Recognition

Standard multi-modal models assume the use of the same modalities in training and inference stages. However, in practice, the environment in which multi-modal models operate may not satisfy such assumption. As such, their performances degrade drastically if any modality is missing in the inference stage. We ask: how can we train a model that is robust to missing modalities? This paper seeks a set of good practices for multi-modal action recognition, with a particular interest in circumstances where some modalities are not available at an inference time. First, we study how to effectively regularize the model during training (e.g., data augmentation). Second, we investigate on fusion methods for robustness to missing modalities: we find that transformer-based fusion shows better robustness for missing modality than summation or concatenation. Third, we propose a simple modular network, ActionMAE, which learns missing modality predictive coding by randomly dropping modality features and tries to reconstruct them with the remaining modality features. Coupling these good practices, we build a model that is not only effective in multi-modal action recognition but also robust to modality missing. Our model achieves the state-of-the-arts on multiple benchmarks and maintains competitive performances even in missing modality scenarios. Codes are available at https://github.com/sangminwoo/ActionMAE.Comment: AAAI 202

Trabecular structural difference between the superior and inferior regions of the vertebral body: a cadaveric and clinical study

BackgroundOsteoporotic vertebral compression fractures commonly involve the superior vertebral body; however, their associated causes have not yet been clearly established. This study aimed to determine the trabecular structural differences between the superior and inferior regions of the vertebral body using cadaveric and clinical studies.Materials and methodsFirst, five vertebrae were collected from three human cadavers. The trabecular structures of the superior and inferior regions of each vertebral body were analyzed using micro-computed tomography (micro-CT), finite element analysis (FEA), and biomechanical test. Based on the results of the ex vivo study, we conducted a clinical study. Second, spine CT images were retrospectively collected. Bone volume and Hounsfield unit were analyzed for 192 vertebral bodies. Finally, after sample size calculation based on the pilot study, prospectively, 200 participants underwent dual-energy X-ray absorptiometry (DXA) of the lateral spine. The bone mineral densities (BMDs) of the superior and inferior regions of each lumbar vertebral body were measured. The paired t-test and Wilcoxon signed-rank test were used for the statistical analyses, and p-value < 0.05 was considered significant.ResultsCadaver studies revealed differences between the superior and inferior trabecular bone structures. The bone volume ratio, BMD, and various other trabecular parameters advocated for decreased strength of the superior region. Throughout the biomechanical study, the limitations of the compression force were 3.44 and 4.63 N/m2 for the superior and inferior regions, respectively. In the FEA study, the inferior region had a lower average displacement and higher von Mises stress than the superior region. In the clinical spine CT-based bone volume and BMD study, the bone volume was significantly higher in the inferior region than in the superior region. In the lateral spine DXA, the mean BMD of the superior region of vertebral bodies was significantly lower compared with that of the inferior region.ConclusionThe superior trabecular structure of the lumbar vertebral bodies possesses more biomechanical susceptibility compared with the inferior trabecular structure, confirming its dominant role in causing osteoporotic vertebral fractures. Physicians should also focus on the BMD values of the superior region of the vertebral body using lateral spine DXA to evaluate osteoporosis

Enhanced oxygen exchange of perovskite oxide surfaces through strain-driven chemical stabilization

Surface cation segregation and phase separation, of strontium in particular, have been suggested to be the key reason behind the chemical instability of perovskite oxide surfaces and the corresponding performance degradation of solid oxide electrochemical cell electrodes. However, there is no well-established solution for effectively suppressing Sr-related surface instabilities. Here, we control the degree of Sr-excess at the surface of SrTi0.5Fe0.5O3-delta thin films, a model mixed conducting perovskite O-2-electrode, through lattice strain, which significantly improves the electrode surface reactivity. Combined theoretical and experimental analyses reveal that Sr cations are intrinsically under a compressive state in the SrTi0.5Fe0.5O3-delta lattice and that the Sr-O bonds are weakened by the local pressure around the Sr cation, which is the key origin of surface Sr enrichment. Based on these findings, we successfully demonstrate that when a large-sized isovalent dopant is added, Sr-excess can be remarkably alleviated, improving the chemical stability of the resulting perovskite O-2-electrodes.11Nsciescopu

The Study of pH Effects on Phase Transition of Multi-Stimuli Responsive P(NiPAAm-co-AAc) Hydrogel Using 2D-COS

The temperature and mechanism of phase transition of poly(N-isopropylacrylamide-co-acrylic acid) [P(NiPAAm-co-AAc)], which is one of the multi-stimuli responsive polymers, were investigated at various pHs using infrared (IR) spectroscopy, two-dimensional (2D) gradient mapping, and two-dimensional correlation spectroscopy (2D-COS). The determined phase transition temperature of P(NiPAAm-co-AAc) at pH 4, 3, and 2 based on 2D gradient mapping and principal component analysis (PCA) showed that it decreases with decreasing pH, because COOH group in AAc changes with variation of pH. The results of 2D-COS analysis indicated that the phase transition mechanism of P(NiPAAm-co-AAc) hydrogel at pH4 is different from that at pH2 due to the effect of COOH group of AAc

Tantalum and molybdenum barriers to prevent carbon diffusion in spark plasma sintered tungsten

Carbide reduces the ductility of tungsten and leads to brittle intergranular fractures. These fractures are inevitably formed in tungsten by carbon diffusion owing to graphite mold during spark plasma sintering. Here, we introduce protective foils made up of two different carbide-forming elements, molybdenum and tantalum, to minimize the carbide formation. Cross-sectional elemental mapping shows that the tantalum foil suppresses carbon diffusion into tungsten, while the molybdenum foil is not an effective diffusion barrier. Thermodynamic-kinetic simulations demonstrate that the suppressed carbon diffusion in tantalum is attributed to high solubility and low diffusivity. Furthermore, the thermodynamically stable tantalum carbide prevents further carbon diffusion at the tantalum/tungsten interface. For the opposite reason, carbon diffuses faster not only in the molybdenum, but also at the molybdenum/tungsten interface. This study provides strategies to minimize the carbon diffusion during spark plasma sintering as well as an intuition into developing structural materials for extreme carbonaceous environments. (C) 2021 Acta Materialia Inc. Published by Elsevier Ltd. All rights reserved.11Nsciescopu

Fabrication of Troponin I Biosensor Composed of Multi-Functional DNA Structure/Au Nanocrystal Using Electrochemical and Localized Surface Plasmon Resonance Dual-Detection Method

In the present study, we fabricated a dual-mode cardiac troponin I (cTnI) biosensor comprised of multi-functional DNA (MF-DNA) on Au nanocrystal (AuNC) using an electrochemical method (EC) and a localized surface plasmon resonance (LSPR) method. To construct a cTnI bioprobe, a DNA 3 way-junction (3WJ) was prepared to introduce multi-functionality. Each DNA 3WJ arm was modified to possess a recognition region (Troponin I detection aptamer), an EC-LSPR signal generation region (methylene blue: MB), and an anchoring region (Thiol group), respectively. After an annealing step, the multi-functional DNA 3WJ was assembled, and its configuration was confirmed by Native-TBM PAGE for subsequent use in biosensor construction. cTnI was also expressed and purified for use in biosensor experiments. To construct an EC-LSPR dual-mode biosensor, AuNCs were prepared on an indium-tin-oxide (ITO) substrate using an electrodeposition method. The prepared multi-functional (MF)-DNA was then immobilized onto AuNCs by covalent bonding. Field emission scanning electron microscope (FE-SEM) and atomic force microscopy (AFM) were used to analyze the surface morphology. LSPR and electrochemical impedance spectroscopy (EIS) experiments were performed to confirm the binding between the target and the bioprobe. The results indicated that cTnI could be effectively detected in the buffer solution and in diluted-human serum. Based on the results of these experiments, the loss on drying (LOD) was determined to be 1.0 pM in HEPES solution and 1.0 pM in 10% diluted human serum. Additionally, the selectivity assay was successfully tested using a number of different proteins. Taken together, the results of our study indicate that the proposed dual-mode biosensor is applicable for use in field-ready cTnI diagnosis systems for emergency situations.Y

Fabrication of Troponin I Biosensor Composed of Multi-Functional DNA Structure/Au Nanocrystal Using Electrochemical and Localized Surface Plasmon Resonance Dual-Detection Method

In the present study, we fabricated a dual-mode cardiac troponin I (cTnI) biosensor comprised of multi-functional DNA (MF-DNA) on Au nanocrystal (AuNC) using an electrochemical method (EC) and a localized surface plasmon resonance (LSPR) method. To construct a cTnI bioprobe, a DNA 3 way-junction (3WJ) was prepared to introduce multi-functionality. Each DNA 3WJ arm was modified to possess a recognition region (Troponin I detection aptamer), an EC-LSPR signal generation region (methylene blue: MB), and an anchoring region (Thiol group), respectively. After an annealing step, the multi-functional DNA 3WJ was assembled, and its configuration was confirmed by Native-TBM PAGE for subsequent use in biosensor construction. cTnI was also expressed and purified for use in biosensor experiments. To construct an EC-LSPR dual-mode biosensor, AuNCs were prepared on an indium-tin-oxide (ITO) substrate using an electrodeposition method. The prepared multi-functional (MF)-DNA was then immobilized onto AuNCs by covalent bonding. Field emission scanning electron microscope (FE-SEM) and atomic force microscopy (AFM) were used to analyze the surface morphology. LSPR and electrochemical impedance spectroscopy (EIS) experiments were performed to confirm the binding between the target and the bioprobe. The results indicated that cTnI could be effectively detected in the buffer solution and in diluted-human serum. Based on the results of these experiments, the loss on drying (LOD) was determined to be 1.0 pM in HEPES solution and 1.0 pM in 10% diluted human serum. Additionally, the selectivity assay was successfully tested using a number of different proteins. Taken together, the results of our study indicate that the proposed dual-mode biosensor is applicable for use in field-ready cTnI diagnosis systems for emergency situations.Y