Bayesian Nested Neural Networks for Uncertainty Calibration and Adaptive Compression

Nested networks or slimmable networks are neural networks whose architectures can be adjusted instantly during testing time, e.g., based on computational constraints. Recent studies have focused on a “nested dropout” layer, which is able to order the nodes of a layer by importance during training, thus generating a nested set of sub-networks that are optimal for different configurations of resources. However, the dropout rate is fixed as a hyperparameter over different layers during the whole training process. Therefore, when nodes are removed, the performance decays in a human-specified trajectory rather than in a trajectory learned from data. Another drawback is the generated sub-networks are deterministic networks without well-calibrated uncertainty. To address these two problems, we develop a Bayesian approach to nested neural networks. We propose a variational ordering unit that draws samples for nested dropout at a low cost, from a proposed Downhill distribution, which provides useful gradients to the parameters of nested dropout. Based on this approach, we design a Bayesian nested neural network that learns the order knowledge of the node distributions. In experiments, we show that the proposed approach outperforms the nested network in terms of accuracy, calibration, and out-of-domain detection in classification tasks. It also outperforms the related approach on uncertainty-critical tasks in computer vision

Keratin 6a marks mammary bipotential progenitor cells that can give rise to a unique tumor model resembling human normal-like breast cancer.

Progenitor cells are considered an important cell of origin of human malignancies. However, there has not been any single gene that can define mammary bipotential progenitor cells, and as such it has not been possible to use genetic methods to introduce oncogenic alterations into these cells in vivo to study tumorigenesis from them. Keratin 6a is expressed in a subset of mammary luminal epithelial cells and body cells of terminal end buds. By generating transgenic mice using the Keratin 6a (K6a) gene promoter to express tumor virus A (tva), which encodes the receptor for avian leukosis virus subgroup A (ALV/A), we provide direct evidence that K6a(+) cells are bipotential progenitor cells, and the first demonstration of a non-basal location for some biopotential progenitor cells. These K6a(+) cells were readily induced to form mammary tumors by intraductal injection of RCAS (an ALV/A-derived vector) carrying the gene encoding the polyoma middle T antigen. Tumors in this K6a-tva line were papillary and resembled the normal breast-like subtype of human breast cancer. This is the first model of this subtype of human tumors and thus may be useful for preclinical testing of targeted therapy for patients with normal-like breast cancer. These observations also provide direct in vivo evidence for the hypothesis that the cell of origin affects mammary tumor phenotypes

Variational Nested Dropout.

Nested dropout is a variant of dropout operation that is able to order network parameters or features based on the pre-defined importance during training. It has been explored for: I. Constructing nested nets Cui et al. 2020, Cui et al. 2021: the nested nets are neural networks whose architectures can be adjusted instantly during testing time, e.g., based on computational constraints. The nested dropout implicitly ranks the network parameters, generating a set of sub-networks such that any smaller sub-network forms the basis of a larger one. II. Learning ordered representation Rippel et al. 2014: the nested dropout applied to the latent representation of a generative model (e.g., auto-encoder) ranks the features, enforcing explicit order of the dense representation over dimensions. However, the dropout rate is fixed as a hyper-parameter during the whole training process. For nested nets, when network parameters are removed, the performance decays in a human-specified trajectory rather than in a trajectory learned from data. For generative models, the importance of features is specified as a constant vector, restraining the flexibility of representation learning. To address the problem, we focus on the probabilistic counterpart of the nested dropout. We propose a variational nested dropout (VND) operation that draws samples of multi-dimensional ordered masks at a low cost, providing useful gradients to the parameters of nested dropout. Based on this approach, we design a Bayesian nested neural network that learns the order knowledge of the parameter distributions. We further exploit the VND under different generative models for learning ordered latent distributions. In experiments, we show that the proposed approach outperforms the nested network in terms of accuracy, calibration, and out-of-domain detection in classification tasks. It also outperforms the related generative models on data generation tasks

Sulforaphane Protects the Liver against CdSe Quantum Dot-Induced Cytotoxicity.

The potential cytotoxicity of cadmium selenide (CdSe) quantum dots (QDs) presents a barrier to their use in biomedical imaging or as diagnostic and therapeutic agents. Sulforaphane (SFN) is a chemoprotective compound derived from cruciferous vegetables which can up-regulate antioxidant enzymes and induce apoptosis and autophagy. This study reports the effects of SFN on CdSe QD-induced cytotoxicity in immortalised human hepatocytes and in the livers of mice. CdSe QDs induced dose-dependent cell death in hepatocytes with an IC50 = 20.4 μM. Pre-treatment with SFN (5 μM) increased cell viability in response to CdSe QDs (20 μM) from 49.5 to 89.3%. SFN induced a pro-oxidant effect characterized by depletion of intracellular reduced glutathione during short term exposure (3-6 h), followed by up-regulation of antioxidant enzymes and glutathione levels at 24 h. SFN also caused Nrf2 translocation into the nucleus, up-regulation of antioxidant enzymes and autophagy. siRNA knockdown of Nrf2 suggests that the Nrf2 pathway plays a role in the protection against CdSe QD-induced cell death. Wortmannin inhibition of SFN-induced autophagy significantly suppressed the protective effect of SFN on CdSe QD-induced cell death. Moreover, the role of autophagy in SFN protection against CdSe QD-induced cell death was confirmed using mouse embryonic fibroblasts lacking ATG5. CdSe QDs caused significant liver damage in mice, and this was decreased by SFN treatment. In conclusion, SFN attenuated the cytotoxicity of CdSe QDs in both human hepatocytes and in the mouse liver, and this protection was associated with the induction of Nrf2 pathway and autophagy

Metallic icosahedron phase of sodium at terapascal pressures

Alkali metals exhibit unexpected structures and electronic behavior at high pressures. Compression of metallic sodium (Na) to 200 GPa leads to the stability of a wide-band-gap insulator with the double hexagonal hP4 structure. Post-hP4 structures remain unexplored, but they are important for addressing the question of the pressure at which Na reverts to a metal. Here, we report the reentrant metallicity of Na at the very high pressure of 15.5 terapascal (TPa), predicted using first-principles structure searching simulations. Na is therefore insulating over the large pressure range of 0.2-15.5 TPa. Unusually, Na adopts an oP8 structure at pressures of 117-125 GPa and the same oP8 structure at 1.75-15.5 TPa. The metallization of Na occurs on the formation of a stable and striking body-centered cubic cI24 electride structure consisting of Na_{12} icosahedra, each housing at its center about one electron that is not associated with any Na ions

Thickness dependence of microstructures in La0.8Ca0.2MnO3 thin films

The thickness dependence of microstructures of La0.8Ca0.2MnO3 (LCMO)/SrTiO3 (STO) thin films was investigated by high-resolution x-ray diffraction, small angle x-ray reflection, grazing incidence x-ray diffraction, scanning electron microscopy, and atomic force microscopy. The results show that all the LCMO films are well oriented in (00l) direction perpendicular to the substrate surface. Self-organized crystalline grains with a tetragonal shape are uniformly distributed on the film surface, indicating the deposition condition being of benefit to the formation of the crystalline grains. With increasing the film thickness, the crystalline quality of the LCMO film is improved, while the surface becomes rougher. There exists a nondesigned cap layer on the upper surface of the LCMO layer for all the samples. The mechanism is discussed briefly.published_or_final_versio

New gold nanostructures for sensor applications: a review

Gold based structures such as nanoparticles (NPs) and nanowires (NWs) have widely been used as building blocks for sensing devices in chemistry and biochemistry fields because of their unusual optical, electrical and mechanical properties. This article gives a detailed review of the new properties and fabrication methods for gold nanostructures, especially gold nanowires (GNWs), and recent developments for their use in optical and electrochemical sensing tools, such as surface enhanced Raman spectroscopy (SERS). © 2014 by the authors; licensee MDPI, Basel, Switzerland

Application of a Common Data Model (CDM) to rank the paediatric user and prescription prevalence of 15 different drug classes in South Korea, Hong Kong, Taiwan, Japan and Australia: an observational, descriptive study

Objective: To measure the paediatric user and prescription prevalence in inpatient and ambulatory settings in South Korea, Hong Kong, Taiwan, Japan and Australia by age and gender. A further objective was to list the most commonly used drugs per drug class, per country. Design and setting: Hospital inpatient and insurance paediatric healthcare data from the following databases were used to conduct this descriptive drug utilisation study: (i) the South Korean Ajou University School of Medicine database; (ii) the Hong Kong Clinical Data Analysis and Reporting System; (iii) the Japan Medical Data Center; (iv) Taiwan’s National Health Insurance Research Database and (v) the Australian Pharmaceutical Benefits Scheme. Country-specific data were transformed into the Observational Medical Outcomes Partnership Common Data Model. Patients: Children (≤18 years) with at least 1 day of observation in any of the respective databases from January 2009 until December 2013 were included. Main outcome measures: For each drug class, we assessed the per-protocol overall user and prescription prevalence rates (per 1000 persons) per country and setting. Results: Our study population comprised 1 574 524 children (52.9% male). The highest proportion of dispensings was recorded in the youngest age category (<2 years) for inpatients (45.1%) with a relatively high user prevalence of analgesics and antibiotics. Adrenergics, antihistamines, mucolytics and corticosteroids were used in 10%–15% of patients. For ambulatory patients, the highest proportion of dispensings was recorded in the middle age category (2–11 years, 67.1%) with antibiotics the most dispensed drug overall. Conclusions: Country-specific paediatric drug utilisation patterns were described, ranked and compared between four East Asian countries and Australia. The widespread use of mucolytics in East Asia warrants further investigation

Urokinase plasminogen activator secreted by cancer-associated fibroblasts induces tumor progression via PI3K/AKT and ERK signaling in esophageal squamous cell carcinoma

published_or_final_versio