138 research outputs found
Improve Generalization and Robustness of Neural Networks via Weight Scale Shifting Invariant Regularizations
Using weight decay to penalize the L2 norms of weights in neural networks has
been a standard training practice to regularize the complexity of networks. In
this paper, we show that a family of regularizers, including weight decay, is
ineffective at penalizing the intrinsic norms of weights for networks with
positively homogeneous activation functions, such as linear, ReLU and
max-pooling functions. As a result of homogeneity, functions specified by the
networks are invariant to the shifting of weight scales between layers. The
ineffective regularizers are sensitive to such shifting and thus poorly
regularize the model capacity, leading to overfitting. To address this
shortcoming, we propose an improved regularizer that is invariant to weight
scale shifting and thus effectively constrains the intrinsic norm of a neural
network. The derived regularizer is an upper bound for the input gradient of
the network so minimizing the improved regularizer also benefits the
adversarial robustness. Residual connections are also considered and we show
that our regularizer also forms an upper bound to input gradients of such a
residual network. We demonstrate the efficacy of our proposed regularizer on
various datasets and neural network architectures at improving generalization
and adversarial robustness.Comment: 14 pages, 5 figure
Bayesian Nested Neural Networks for Uncertainty Calibration and Adaptive Compression
Nested networks or slimmable networks are neural networks whose architectures
can be adjusted instantly during testing time, e.g., based on computational
constraints. Recent studies have focused on a "nested dropout" layer, which is
able to order the nodes of a layer by importance during training, thus
generating a nested set of sub-networks that are optimal for different
configurations of resources. However, the dropout rate is fixed as a
hyper-parameter over different layers during the whole training process.
Therefore, when nodes are removed, the performance decays in a human-specified
trajectory rather than in a trajectory learned from data. Another drawback is
the generated sub-networks are deterministic networks without well-calibrated
uncertainty. To address these two problems, we develop a Bayesian approach to
nested neural networks. We propose a variational ordering unit that draws
samples for nested dropout at a low cost, from a proposed Downhill
distribution, which provides useful gradients to the parameters of nested
dropout. Based on this approach, we design a Bayesian nested neural network
that learns the order knowledge of the node distributions. In experiments, we
show that the proposed approach outperforms the nested network in terms of
accuracy, calibration, and out-of-domain detection in classification tasks. It
also outperforms the related approach on uncertainty-critical tasks in computer
vision.Comment: 16 pages, 10 figure
An inverted J-shaped association of serum uric acid with muscle strength among Japanese adult men: a cross-sectional study
BACKGROUND: Uric acid (UA) may protect muscle function from oxidative damage due to reactive oxygen species through its powerful antioxidant capacity. However, several studies have demonstrated that hyperuricemia is closely related to systemic inflammation and has oxidant properties effects, both of which may increase the risk of muscle strength loss. The purpose of this study was to examine the association of serum UA concentration with grip strength and leg extension power in adult men. METHODS: This study is a cross-sectional survey in which 630 Japanese male employees aged 30Â years and older participated. Five hundred and eighty-six subjects participated in the measurement of grip strength, and 355 subjects participated in the measurement of leg extension power. Blood samples were obtained for serum UA analysis. RESULTS: After adjustment for potential confounders, grip strength differed significantly between participants with and those without hyperuricemia (geometric mean and 95% confidence interval [CI]: 40.3 [39.2â41.3] kg vs. 41.9 [41.3â42.5] kg; Pâ=â0.01). In addition, serum UA levels (quartiles) showed an inverted J-shaped curve with grip strength (mean and 95% CI: Q1, 41.6 [40.6â42.6] kg; Q2, 42.2 [41.2â43.2] kg; Q3, 41.8 [40.8â42.8] kg; Q4, 40.4 [39.3â41.4] kg; P for quadratic trendâ=â0.05). The results in the leg extension power group were similar to those observed in the grip strength group. CONCLUSION: This population-based cross-sectional study shows for the first time that hyperuricemia is associated with poor muscle strength. Moreover, the results indicate an inverted J-shaped association between serum UA quartiles and muscle strength
Changes of predominant species/biovars and sequence types of Brucellaisolates, Inner Mongolia, China
BACKGROUND: Human brucellosis incidence in China was divided into 3 stages, high incidence (1950-1960s), decline (1970-1980s) and re-emergence (1990-2000s). Human brucellosis has been reported in all the 32 provinces, of which Inner Mongolia has the highest prevalence, accounting for over 40% of the cases in China. To investigate the etiology alteration of human brucellosis in Inner Mongolia, the species, biovars and genotypes of 60 Brucella isolates from this province were analyzed. METHODS: Species and biovars of the Brucella strains isolated from outbreaks were determined based on classical identification procedures. Strains were genotyped by multi locus sequence typing (MLST). Sequences of 9 housekeeping genes were obtained and sequence types were defined. The distribution of species, biovars and sequence types (STs) among the three incidence stages were analyzed and compared. RESULTS: The three stages of high incidence, decline and re-emergence were predominated by B. melitensis biovar 2 and 3, B. abortus biovar 3, and B. melitensis biovar 1, respectively, implying changes in the predominant biovars. Genotyping by MLST revealed a total of 14 STs. Nine STs (from ST28 to ST36), accounting for 64.3% of all the STs, were newly defined and different from those observed in other countries. Different STs were distributed among the three stages. ST8 was the most common ST in 1950-1960s and 1990-2000s, while ST2 was the most common in 1970-1980s. CONCLUSIONS: The prevalence of biovars and sequence types of Brucella strains from Inner Mongolia has changed over time in the three stages. Compared with those from other countries, new sequence types of Brucella strains exist in China
Euryale Ferox Seed-inspired Super-lubricated Nanoparticles for Treatment of Osteoarthritis
Osteoarthritis has been regarded as a typical lubrication deficiency related joint disease, which is characterized by the breakdown of articular cartilage at the joint surface and the inflammation of the joint capsule. Here, inspired by the structure of the fresh euryale ferox seed that possesses a slippery aril and a hard coat containing starchy kernel, a novel superlubricated nanoparticle, namely poly (3âsulfopropyl methacrylate potassium salt)âgrafted mesoporous silica nanoparticles (MSNsâNH2@PSPMK), is biomimicked and synthesized via a oneâstep photopolymerization method. The nanoparticles are endowed with enhanced lubrication by the grafted PSPMK polyelectrolyte polymer due to the formation of tenacious hydration layers surrounding the negative charges, and simultaneously are featured with effective drug loading and release behavior as a result of the sufficient mesoporous channels in the MSNs. When encapsulated with an antiâinflammatory drug diclofenac sodium (DS), the lubrication capability of the superlubricated nanoparticles is improved, while the drug release rate is sustained by increasing the thickness of PSPMK layer, which is simply achieved via adjustment of the precursor monomer concentration in the photopolymerization process. Additionally, the in vitro and in vivo experimental results show that the DSâloaded MSNsâNH2@PSPMK nanoparticles effectively protect the chondrocytes from degeneration, and thus, inhibit the development of osteoarthritis.Peer reviewe
Droplet spatial distribution of oil-based emulsion spray
IntroductionOil-based emulsion solution is a common pesticide formulation in agricultural spraying, and its spray characteristics are different from that of water spraying. The well understanding of its spray characteristics is the theoretical basis to improve the pesticide spraying technology. The objective of the present study is to deepen the understanding of the spray characteristics of oil-based emulsion.MethodIn this paper, the spatial distribution characteristics of spray droplets of oil-based emulsion were captured visually using the high-speed photomicrography. On the basis of image processing method, the droplet size and distribution density of spray droplets at different spatial locations were analyzed quantitatively. The effects of nozzle configuration and emulsion concentration on spray structures and droplet spatial distribution were discussed.ResultsOil-based emulsion produced a special perforation atomization mechanism compared with water spray, which led to the increase of spray droplet size and distribution density. Nozzle configuration had a significant effect on oil-based emulsion spray, with the nozzle changed from ST110-01 to ST110-03 and ST110-05; the sheet lengths increased to 18 and 28Â mm, respectively, whereas the volumetric median diameters increased to 51.19% and 76.00%, respectively. With emulsion concentration increased from 0.02% to 0.1% and 0.5%, the volumetric median diameters increased to 5.17% and 14.56%, respectively.DiscussionThe spray droplet size of oil-based emulsion spray can be scaled by the equivalent diameter of discharge orifice of nozzles. The products of volumetric median diameters and corresponding surface tensions were nearly constant for the oil-based emulsion spray of different emulsion concentrations. It is expected that this research could provide theoretical support for improving the spraying technology of oil-based emulsion and increasing the utilization of pesticide
A scoping review of utilization of the verbal fluency task in Chinese and Japanese clinical settings with near-infrared spectroscopy
This review targets the application of the Verbal Fluency Task (VFT) in conjunction with functional near-infrared spectroscopy (fNIRS) for diagnosing psychiatric disorders, specifically in the contexts of China and Japan. These two countries are at the forefront of integrating fNIRS with VFT in clinical psychiatry, often employing this combination as a complementary tool alongside traditional psychiatric examinations. Our study aims to synthesize research findings on the hemodynamic responses elicited by VFT task in clinical settings of the two countries, analyzing variations in task design (phonological versus semantic), stimulus modality (auditory versus visual), and the impact of language typology. The focus on China and Japan is crucial, as it provides insights into the unique applications and adaptations of VFT in these linguistically and culturally distinct environments. By exploring these specific cases, our review underscores the importance of tailoring VFT to fit the linguistic and cultural context, thereby enhancing its validity and utility in cross-cultural psychiatric assessments
Correlating Chemical Reaction and Mass Transport in Hydrogen-based Direct Reduction of Iron Oxide
Steelmaking contributes 8% to the total CO2 emissions globally, primarily due
to coal-based iron ore reduction. Clean hydrogen-based ironmaking has variable
performance because the dominant gas-solid reduction mechanism is set by the
defects and pores inside the mm-nm sized oxide particles that change
significantly as the reaction progresses. While these governing dynamics are
essential to establish continuous flow of iron and its ores through reactors,
the direct link between agglomeration and chemistry is still contested due to
missing measurements. In this work, we directly measure the connection between
chemistry and agglomeration in the smallest iron oxides relevant to magnetite
ores. Using synthesized spherical 10-nm magnetite particles reacting in H2, we
resolve the formation and consumption of w\"ustite (FeO) - the step most
commonly attributed to agglomeration. Using X-ray scattering and microscopy, we
resolve crystallographic anisotropy in the rate of the initial reaction, which
becomes isotropic as the material sinters. Complementing with imaging, we
demonstrate how the particles self-assemble, subsequently react and sinter into
~100x oblong grains. Our insights into how morphologically uniform iron oxide
particles react and agglomerate H2 reduction enable future size-dependent
models to effectively describe the multiscale iron ore reduction
- âŠ