Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees

Deep Reinforcement Learning (DRL) has achieved impressive success in many applications. A key component of many DRL models is a neural network representing a Q function, to estimate the expected cumulative reward following a state-action pair. The Q function neural network contains a lot of implicit knowledge about the RL problems, but often remains unexamined and uninterpreted. To our knowledge, this work develops the first mimic learning framework for Q functions in DRL. We introduce Linear Model U-trees (LMUTs) to approximate neural network predictions. An LMUT is learned using a novel on-line algorithm that is well-suited for an active play setting, where the mimic learner observes an ongoing interaction between the neural net and the environment. Empirical evaluation shows that an LMUT mimics a Q function substantially better than five baseline methods. The transparent tree structure of an LMUT facilitates understanding the network's learned knowledge by analyzing feature influence, extracting rules, and highlighting the super-pixels in image inputs.Comment: This paper is accepted by ECML-PKDD 201

Decrease in water clarity of the southern and central North Sea during the 20th century

Light in the marine environment is a key environmental variable coupling physics to marine biogeochemistry and ecology. Weak light penetration reduces light available for photosynthesis, changing energy fluxes through the marine food web. Based on published and unpublished data, this study shows that the central and southern North Sea has become significantly less clear over the second half of the 20th century. In particular, in the different regions and seasons investigated, the average Secchi depth pre-1950 decreased between 25% and 75% compared to the average Secchi depth post-1950. Consequently, in summer pre-1950, most (74%) of the sea floor in the permanently mixed area off East Anglia was within the photic zone. For the last 25+ years, changes in water clarity were more likely driven by an increase in the concentration of suspended sediments, rather than phytoplankton. We suggest that a combination of causes have contributed to this increase in suspended sediments such as changes in sea-bed communities and in weather patterns, decreased sink of sediments in estuaries, and increased coastal erosion. A predicted future increase in storminess (Beniston et al., 2007; Kovats et al., 2014) could enhance the concentration of suspended sediments in the water column and consequently lead to a further decrease in clarity, with potential impacts on phytoplankton production, CO2 fluxes, and fishery production

Soft eSkin:distributed touch sensing with harmonized energy and computing

Inspired by biology, significant advances have been made in the field of electronic skin (eSkin) or tactile skin. Many of these advances have come through mimicking the morphology of human skin and by distributing few touch sensors in an area. However, the complexity of human skin goes beyond mimicking few morphological features or using few sensors. For example, embedded computing (e.g. processing of tactile data at the point of contact) is centric to the human skin as some neuroscience studies show. Likewise, distributed cell or molecular energy is a key feature of human skin. The eSkin with such features, along with distributed and embedded sensors/electronics on soft substrates, is an interesting topic to explore. These features also make eSkin significantly different from conventional computing. For example, unlike conventional centralized computing enabled by miniaturized chips, the eSkin could be seen as a flexible and wearable large area computer with distributed sensors and harmonized energy. This paper discusses these advanced features in eSkin, particularly the distributed sensing harmoniously integrated with energy harvesters, storage devices and distributed computing to read and locally process the tactile sensory data. Rapid advances in neuromorphic hardware, flexible energy generation, energy-conscious electronics, flexible and printed electronics are also discussed. This article is part of the theme issue ‘Harmonizing energy-autonomous computing and intelligence’

Weakly Supervised Learning by a Confusion Matrix of Contexts

© 2019, Springer Nature Switzerland AG. Context consideration can help provide more background and related information for weakly supervised learning. The inclusion of less documented historical and environmental context in researching diabetes amongst Pima Indians uncovered reasons which were more likely to explain why some Pima Indians had much higher rates of diabetes than Caucasians, primarily due to historical, environmental and social causes rather than their specific genetic patterns or ethnicity as suggested by many medical studies. If historical and environmental factors are considered as external contexts when not included as part of a dataset for research, some forms of internal contexts may also exist inside the dataset without being declared. This paper discusses a context construction model that transforms a confusion matrix into a matrix of categorical, incremental and correlational context to emulate a kind of internal context to search for more informative patterns in order to improve weakly supervised learning from limited labeled samples for unlabeled data. When the negative and positive labeled samples and misclassification errors are compared to “happy families” and “unhappy families”, the contexts constructed by this model in the classification experiments reflected the Anna Karenina principle well - “Happy families are all alike; every unhappy family is unhappy in its own way”, an encouraging sign to further explore contexts associated with harmonizing patterns and divisive causes for knowledge discovery in a world of uncertainty

Yukawa Unification and the Superpartner Mass Scale

Naturalness in supersymmetry (SUSY) is under siege by increasingly stringent LHC constraints, but natural electroweak symmetry breaking still remains the most powerful motivation for superpartner masses within experimental reach. If naturalness is the wrong criterion then what determines the mass scale of the superpartners? We motivate supersymmetry by (1) gauge coupling unification, (2) dark matter, and (3) precision b-tau Yukawa unification. We show that for an LSP that is a bino-Higgsino admixture, these three requirements lead to an upper-bound on the stop and sbottom masses in the several TeV regime because the threshold correction to the bottom mass at the superpartner scale is required to have a particular size. For tan beta about 50, which is needed for t-b-tau unification, the stops must be lighter than 2.8 TeV when A_t has the opposite sign of the gluino mass, as is favored by renormalization group scaling. For lower values of tan beta, the top and bottom squarks must be even lighter. Yukawa unification plus dark matter implies that superpartners are likely in reach of the LHC, after the upgrade to 14 (or 13) TeV, independent of any considerations of naturalness. We present a model-independent, bottom-up analysis of the SUSY parameter space that is simultaneously consistent with Yukawa unification and the hint for m_h = 125 GeV. We study the flavor and dark matter phenomenology that accompanies this Yukawa unification. A large portion of the parameter space predicts that the branching fraction for B_s to mu^+ mu^- will be observed to be significantly lower than the SM value.Comment: 34 pages plus appendices, 20 figure

Using Bayesian Networks and Machine Learning to Predict Computer Science Success

Bayesian Networks and Machine Learning techniques were evaluated and compared for predicting academic performance of Computer Science students at the University of Cape Town. Bayesian Networks performed similarly to other classification models. The causal links AQ1 inherent in Bayesian Networks allow for understanding of the contributing factors for academic success in this field. The most effective indicators of success in first-year ‘core’ courses in Computer Science included the student’s scores for Mathematics and Physics as well as their aptitude for learning and their work ethos. It was found that unsuccessful students could be identified with ≈91% accuracy. This could help to increase throughput as well as student wellbeing at university

Implications of large dimuon CP asymmetry in B_{d,s} decays on minimal flavor violation with low tan beta

The D0 collaboration has recently announced evidence for a dimuon CP asymmetry in B_{d,s} decays of order one percent. If confirmed, this asymmetry requires new physics. We argue that for minimally flavor violating (MFV) new physics, and at low tan beta=v_u/v_d, there are only two four-quark operators (Q_{2,3}) that can provide the required CP violating effect. The scale of such new physics must lie below 260 GeV sqrt{tan beta}. The effect is universal in the B_s and B_d systems, leading to S_{psi K}~sin(2beta)-0.15 and S_{psi phi}~0.25. The effects on epsilon_K and on electric dipole moments are negligible. The most plausible mechanism is tree-level scalar exchange. MFV supersymmetry with low tan beta will be excluded. Finally, we explain how a pattern of deviations from the Standard Model predictions for S_{psi phi}, S_{psi K} and epsilon_K can be used to test MFV and, if MFV holds, to probe its structure in detail.Comment: 11 pages. v2: References adde

Using technology to deliver cancer follow-up : a systematic review

Peer reviewedPublisher PD

Minimal flavour violation extensions of the seesaw

We analyze the most natural formulations of the minimal lepton flavour violation hypothesis compatible with a type-I seesaw structure with three heavy singlet neutrinos N, and satisfying the requirement of being predictive, in the sense that all LFV effects can be expressed in terms of low energy observables. We find a new interesting realization based on the flavour group $SU(3)_e\times SU(3)_{\ell+N}$ (being $e$ and $\ell$ respectively the SU(2) singlet and doublet leptons). An intriguing feature of this realization is that, in the normal hierarchy scenario for neutrino masses, it allows for sizeable enhancements of $\mu \to e$ transitions with respect to LFV processes involving the $\tau$ lepton. We also discuss how the symmetries of the type-I seesaw allow for a strong suppression of the N mass scale with respect to the scale of lepton number breaking, without implying a similar suppression for possible mechanisms of N productionComment: 14 pages, 6 figure