Improving Interpretability and Regularization in Deep Learning

IEEE Deep learning approaches yield state-of-the-art performance in a range of tasks, including automatic speech recognition. However, the highly distributed representation in a deep neural network (DNN) or other network variations are difficult to analyse, making further parameter interpretation and regularisation challenging. This paper presents a regularisation scheme acting on the activation function output to improve the network interpretability and regularisation. The proposed approach, referred to as activation regularisation, encourages activation function outputs to satisfy a target pattern. By defining appropriate target patterns, different learning concepts can be imposed on the network. This method can aid network interpretability and also has the potential to reduce over-fitting. The scheme is evaluated on several continuous speech recognition tasks: the Wall Street Journal continuous speech recognition task, eight conversational telephone speech tasks from the IARPA Babel program and a U.S. English broadcast news task. On all the tasks, the activation regularisation achieved consistent performance gains over the standard DNN baselines

Cell voltage versus electrode potential range in aqueous supercapacitors

Supercapacitors with aqueous electrolytes and nanostructured composite electrodes are attractive because of their high charging-discharging speed, long cycle life, low environmental impact and wide commercial affordability. However, the energy capacity of aqueous supercapacitors is limited by the electrochemical window of water. In this paper, a recently reported engineering strategy is further developed and demonstrated to correlate the maximum charging voltage of a supercapacitor with the capacitive potential ranges and the capacitance ratio of the two electrodes. Beyond the maximum charging voltage, a supercapacitor may still operate, but at the expense of a reduced cycle life. In addition, it is shown that the supercapacitor performance is strongly affected by the initial and zero charge potentials of the electrodes. Further, the differences are highlighted and elaborated between freshly prepared, aged under open circuit conditions, and cycled electrodes of composites of conducting polymers and carbon nanotubes. The first voltammetric charging-discharging cycle has an electrode conditioning effect to change the electrodes from their initial potentials to the potential of zero voltage, and reduce the irreversibility

Subcellular localization of MC4R with ADCY3 at neuronal primary cilia underlies a common pathway for genetic predisposition to obesity.

Most monogenic cases of obesity in humans have been linked to mutations in genes encoding members of the leptin-melanocortin pathway. Specifically, mutations in MC4R, the melanocortin-4 receptor gene, account for 3-5% of all severe obesity cases in humans1-3. Recently, ADCY3 (adenylyl cyclase 3) gene mutations have been implicated in obesity4,5. ADCY3 localizes to the primary cilia of neurons 6 , organelles that function as hubs for select signaling pathways. Mutations that disrupt the functions of primary cilia cause ciliopathies, rare recessive pleiotropic diseases in which obesity is a cardinal manifestation 7 . We demonstrate that MC4R colocalizes with ADCY3 at the primary cilia of a subset of hypothalamic neurons, that obesity-associated MC4R mutations impair ciliary localization and that inhibition of adenylyl cyclase signaling at the primary cilia of these neurons increases body weight. These data suggest that impaired signaling from the primary cilia of MC4R neurons is a common pathway underlying genetic causes of obesity in humans

How do modern transportation projects impact on development of impervious surfaces via new urban area and urban intensification? Evidence from Hangzhou Bay Bridge, China

Many countries have been constructing modern ground transportation projects. This raises questions about the impacts of such projects on development of impervious surfaces, yet there have been few attempts to systematically analyze these impacts. This paper attempts to narrow this information gap using the Hangzhou Bay Bridge project, China, as an exploratory case study. Using remotely sensed data, we developed a framework based on statistical techniques, wavelet multi-resolution analysis and Theil-Sen slope analysis to measure the changes in impervious surfaces. The derived changes were then linked to the bridge project with respect to socio-economic factors and land use development activities. The findings highlight that the analytical framework could reliably quantify the area, pattern and form of new urban area and urban intensification. Change detection analysis showed that urban area, GDP and the length of highways increased moderately in the pre-Hangzhou Bay Bridge period (1995–2002) while all of these variables increased more substantially during (2002–2009) and after (2009–2013) the bridge construction. The results indicate that the development of impervious surfaces due to new urban area came at the expense of permeable surfaces in the urban fringe and within rural regions, while urban intensification occurred mainly in the form of the redevelopment of older structures to modern high-rise buildings within existing urban regions. In the context of improved transportation infrastructure, our findings suggest that new urban area and urban intensification can be attributed to consecutive events which act like a chain reaction: construction of improved transportation projects, their impacts on land use development policies, effects of both systems on socio-economic variables, and finally all these changes influence new urban area and urban intensification. However, more research is needed to better understand this sequential process and to examine the broader applicability of the concept in other developing regions

Hot Jupiters from Secular Planet--Planet Interactions

About 25 per cent of `hot Jupiters' (extrasolar Jovian-mass planets with close-in orbits) are actually orbiting counter to the spin direction of the star. Perturbations from a distant binary star companion can produce high inclinations, but cannot explain orbits that are retrograde with respect to the total angular momentum of the system. Such orbits in a stellar context can be produced through secular (that is, long term) perturbations in hierarchical triple-star systems. Here we report a similar analysis of planetary bodies, including both octupole-order effects and tidal friction, and find that we can produce hot Jupiters in orbits that are retrograde with respect to the total angular momentum. With distant stellar mass perturbers, such an outcome is not possible. With planetary perturbers, the inner orbit's angular momentum component parallel to the total angular momentum need not be constant. In fact, as we show here, it can even change sign, leading to a retrograde orbit. A brief excursion to very high eccentricity during the chaotic evolution of the inner orbit allows planet-star tidal interactions to rapidly circularize that orbit, decoupling the planets and forming a retrograde hot Jupiter.Comment: accepted for publication by Nature, 3 figures (version after proof - some typos corrected

BOLD Temporal Dynamics of Rat Superior Colliculus and Lateral Geniculate Nucleus following Short Duration Visual Stimulation

Background: The superior colliculus (SC) and lateral geniculate nucleus (LGN) are important subcortical structures for vision. Much of our understanding of vision was obtained using invasive and small field of view (FOV) techniques. In this study, we use non-invasive, large FOV blood oxygenation level-dependent (BOLD) fMRI to measure the SC and LGN's response temporal dynamics following short duration (1 s) visual stimulation. Methodology/Principal Findings: Experiments are performed at 7 tesla on Sprague Dawley rats stimulated in one eye with flashing light. Gradient-echo and spin-echo sequences are used to provide complementary information. An anatomical image is acquired from one rat after injection of monocrystalline iron oxide nanoparticles (MION), a blood vessel contrast agent. BOLD responses are concentrated in the contralateral SC and LGN. The SC BOLD signal measured with gradient-echo rises to 50% of maximum amplitude (PEAK) 0.2±0.2 s before the LGN signal (p<0.05). The LGN signal returns to 50% of PEAK 1.4±1.2 s before the SC signal (p<0.05). These results indicate the SC signal rises faster than the LGN signal but settles slower. Spin-echo results support these findings. The post-MION image shows the SC and LGN lie beneath large blood vessels. This subcortical vasculature is similar to that in the cortex, which also lies beneath large vessels. The LGN lies closer to the large vessels than much of the SC. Conclusions/Significance: The differences in response timing between SC and LGN are very similar to those between deep and shallow cortical layers following electrical stimulation, which are related to depth-dependent blood vessel dilation rates. This combined with the similarities in vasculature between subcortex and cortex suggest the SC and LGN timing differences are also related to depth-dependent dilation rates. This study shows for the first time that BOLD responses in the rat SC and LGN following short duration visual stimulation are temporally different. © 2011 Lau et al

Classification and Analysis of Regulatory Pathways Using Graph Property, Biochemical and Physicochemical Property, and Functional Property

Given a regulatory pathway system consisting of a set of proteins, can we predict which pathway class it belongs to? Such a problem is closely related to the biological function of the pathway in cells and hence is quite fundamental and essential in systems biology and proteomics. This is also an extremely difficult and challenging problem due to its complexity. To address this problem, a novel approach was developed that can be used to predict query pathways among the following six functional categories: (i) “Metabolism”, (ii) “Genetic Information Processing”, (iii) “Environmental Information Processing”, (iv) “Cellular Processes”, (v) “Organismal Systems”, and (vi) “Human Diseases”. The prediction method was established trough the following procedures: (i) according to the general form of pseudo amino acid composition (PseAAC), each of the pathways concerned is formulated as a 5570-D (dimensional) vector; (ii) each of components in the 5570-D vector was derived by a series of feature extractions from the pathway system according to its graphic property, biochemical and physicochemical property, as well as functional property; (iii) the minimum redundancy maximum relevance (mRMR) method was adopted to operate the prediction. A cross-validation by the jackknife test on a benchmark dataset consisting of 146 regulatory pathways indicated that an overall success rate of 78.8% was achieved by our method in identifying query pathways among the above six classes, indicating the outcome is quite promising and encouraging. To the best of our knowledge, the current study represents the first effort in attempting to identity the type of a pathway system or its biological function. It is anticipated that our report may stimulate a series of follow-up investigations in this new and challenging area

iDNA-Prot: Identification of DNA Binding Proteins Using Random Forest with Grey Model

DNA-binding proteins play crucial roles in various cellular processes. Developing high throughput tools for rapidly and effectively identifying DNA-binding proteins is one of the major challenges in the field of genome annotation. Although many efforts have been made in this regard, further effort is needed to enhance the prediction power

Bilateral Assessment of Functional Tasks for Robot-assisted Therapy Applications

This article presents a novel evaluation system along with methods to evaluate bilateral coordination of arm function on activities of daily living tasks before and after robot-assisted therapy. An affordable bilateral assessment system (BiAS) consisting of two mini-passive measuring units modeled as three degree of freedom robots is described. The process for evaluating functional tasks using the BiAS is presented and we demonstrate its ability to measure wrist kinematic trajectories. Three metrics, phase difference, movement overlap, and task completion time, are used to evaluate the BiAS system on a bilateral symmetric (bi-drink) and a bilateral asymmetric (bi-pour) functional task. Wrist position and velocity trajectories are evaluated using these metrics to provide insight into temporal and spatial bilateral deficits after stroke. The BiAS system quantified movements of the wrists during functional tasks and detected differences in impaired and unimpaired arm movements. Case studies showed that stroke patients compared to healthy subjects move slower and are less likely to use their arm simultaneously even when the functional task requires simultaneous movement. After robot-assisted therapy, interlimb coordination spatial deficits moved toward normal coordination on functional tasks