Simplification of rules extracted from neural networks

Artificial neural networks (ANNs) have been proven to be successful general machine learning techniques for, amongst others, pattern recognition and classification. Realworld problems in agriculture (soybean, tea), medicine (cancer, cardiology, mammograms) and finance (credit rating, stock market) are successfully solved using ANNs. ANNs model biological neural systems. A biological neural system consists of neurons interconnected through neural synapses. These neurons serve as information processing units. Synapses carrt information to the neurons, which then processes or responds to the data by sending a signal to the next level of neurons. Information is strengthened or lessened according to the sign ..and magnitude of the weight associated with the connection. An ANN consists of cell-like entities called units (also called artificial neurons) and weighted connections between these units referred to as links. ANNs can be viewed as a directed graph with weighted connections. An unit belongs to one of three groups: input, hidden or output. Input units receive the initial training patterns, which consist of input attributes and the associated target attributes, from the environment. Hidden units do not interact with the environment whereas output units presents the results to the environment. Hidden and output units compute an output ai which is a function f of the sum of its input weights w; multiplied by the output x; of the units j in the preceding layer, together with a bias term fh that acts as a threshold for the unit. The output ai for unit i with n input units is calculated as ai = f("f:,'J= 1 x;w; - 8i ). Training of the ANN is done by adapting the weight values for each unit via a gradient search. Given a set of input-target pairs, the ANN learns the functional relationship between the input and the target. A serious drawback of the neural network approach is the difficulty to determine why a particular conclusion was reached. This is due to the inherit 'black box' nature of the neural network approach. Neural networks rely on 'raw' training data to learn the relationships between the initial inputs and target outputs. Knowledge is encoded in a set of numeric weights and biases. Although this data driven aspect of neural network allows easy adjustments when change of environment or events occur, it is difficult to interpret numeric weights, making it difficult for humans to understand. Concepts represent by symbolic learning algorithms are intuitive and therefore easily understood by humans [Wnek 1994). One approach to understanding the representations formed by neural networks is to extract such symbolic rules from networks. Over the last few years, a number of rule extraction methods have been reported (Craven 1993, Fu 1994). There are some general assumptions that these algorithms adhere to. The first assumption that most rule extraction algorithms make, is that non-input units are either maximally active (activation near 1) or inactive (activation near 0). This Boolean valued activation is approximated by using the standard logistic activation function /(z) = 1/( 1 + e-•z ) and setting s 5.0. The use of the above function parameters guarantees that non-input units always have non-negative activations in the range [0,1). The second underlying premise of rule extraction is that each hidden and output unit implements a symbolic rule. The concept associated with each unit is the consequent of the rule, and certain subsets of the input units represent the antecedent of the rule. Rule extraction algorithms search for those combinations of input values to a particular hidden or output unit that results in it having an optimal (near-one) activation. Here, rule extraction methods exploit a very basic principle of biological neural networks. That is, if the sum of its weighted inputs exceeds a certain threshold, then the biological neuron fires [Fu 1994). This condition is satisfied when the sum of the weighted inputs exceeds the bias, where (E'Jiz,=::l w; > 9i)• It has been shown that most concepts described by humans usally can be expressed as production rules in disjunctive normal form (DNF) notation. Rules expressed in this notation are therefore highly comprehensible and intuitive. In addition, the number of production rules may be reduced and the structure thereof simplified by using propositional logic. A method that extracts production rules in DNF is presented [Viktor 1995). The basic idea of the method is the use of equivalence classes. Similarly weighted links are grouped into a cluster, the assumption being that individual weights do not have unique importance. Clustering considerably reduces the combinatorics of the method as opposed to previously reported approaches. Since the rules are in a logically manipulatable form, significant simplifications in the structure thereof can be obtained, yielding a highly reduced and comprehensible set of rules. Experimental results have shown that the accuracy of the extracted rules compare favourably with the CN2 [Clark 1989] and C4.5 [Quinlan 1993] symbolic rule extraction methods. The extracted rules are highly comprehensible and similar to those extracted by traditional symfiolic methods

An equine tendon model for studying intra-tendinous shear in tendons that have more than one muscle contribution

Human Achilles tendon is composed of three smaller sub-tendons and exhibits non-uniform internal displacements, which decline with age and after injury, suggesting a potential role in the development of tendinopathies. Studying internal sliding behaviour is therefore important but difficult in human Achilles tendon. Here we propose the equine deep digital flexor tendon (DDFT) and its accessory ligament (AL) as a model to understand the sliding mechanism. The AL-DDFT has a comparable sub-bundle structure, is subjected to high and frequent asymmetric loads and is a natural site of injury similar to human Achilles tendons. Equine AL-DDFT were collected and underwent whole tendon level (n=7) and fascicle level (n=7) quasi-static mechanical testing. Whole tendon level testing was performed by sequentially loading through the proximal AL and subsequently through the proximal DDFT and recording regional strain in the free structures and joined DDFT and AL. Fascicle level testing was performed with focus on the inter-sub-bundle matrix between the two structures at the junction. Our results demonstrate a significant difference in the regional strain between the joined DDFT and AL and a greater transmission of force from the AL to the DDFT than vice versa. These results can be partially explained by the mechanical properties and geometry of the two structures and by differences in the properties of the interfascicular matrices. In conclusion, this tendon model successfully demonstrates that high displacement discrepancy occurs between the two structures and can be used as an easy-access model for study intra-tendinous shear mechanics at the sub-tendon level. STATEMENT OF SIGNIFICANCE: : Our study provides a naturally occurring and easily accessible equine model to study the complex behaviour of sub-tendons within the human Achilles tendon, which is likely to play a critical role in the pathogenesis of tendon disease. Our results demonstrate that the difference in material stiffness between the equine AL and DDFT stems largely from differences in the inter-fascicular matrix and furthermore that differences in strain are maintained in distal parts of the tightly joined structure. Furthermore, our results suggest that distribution of load between sub-structures is highly dependent on the morphological relationship between them; a finding that has important implications for understanding Achilles tendon mechanical behaviour, injury mechanisms and rehabilitation

General graviton exchange graph for four point functions in the AdS/CFT correspondence

In this note we explicitly compute the graviton exchange graph for scalar fields with arbitrary conformal dimension \Delta in arbitrary spacetime dimension d. This results in an analytical function in \Delta as well as in d.Comment: 14 pages, 2 figure

Fast Algorithms for Join Operations on Tree Decompositions

Treewidth is a measure of how tree-like a graph is. It has many important algorithmic applications because many NP-hard problems on general graphs become tractable when restricted to graphs of bounded treewidth. Algorithms for problems on graphs of bounded treewidth mostly are dynamic programming algorithms using the structure of a tree decomposition of the graph. The bottleneck in the worst-case run time of these algorithms often is the computations for the so called join nodes in the associated nice tree decomposition. In this paper, we review two different approaches that have appeared in the literature about computations for the join nodes: one using fast zeta and M\"obius transforms and one using fast Fourier transforms. We combine these approaches to obtain new, faster algorithms for a broad class of vertex subset problems known as the [\sigma,\rho]-domination problems. Our main result is that we show how to solve [\sigma,\rho]-domination problems in $O(s^{t+2} t n^2 (t\log(s)+\log(n)))$ arithmetic operations. Here, t is the treewidth, s is the (fixed) number of states required to represent partial solutions of the specific [\sigma,\rho]-domination problem, and n is the number of vertices in the graph. This reduces the polynomial factors involved compared to the previously best time bound (van Rooij, Bodlaender, Rossmanith, ESA 2009) of $O( s^{t+2} (st)^{2(s-2)} n^3 )$ arithmetic operations. In particular, this removes the dependence of the degree of the polynomial on the fixed number of states~$s$.Comment: An earlier version appeared in "Treewidth, Kernels, and Algorithms. Essays Dedicated to Hans L. Bodlaender on the Occasion of His 60th Birthday" LNCS 1216

To what extent can the experience of outdoor learning contexts prevent permanent school exclusion for older learners? A visual analysis

This is the author accepted manuscript. The final version is available from IATED via the DOI in this record.We report on a one-year project that focused on outdoor learning experiences for learners 12 - 14 years of age in a woodland environment in the UK. We wanted to investigate the ways in which experience in the outdoor environment could potentially mitigate school factors such as practitioner values and attitudes, learner motivation and engagement [1] that contribute to the processes of permanent school exclusion and therefore examine the claim that outdoor learning could provide an ‘alternative’ to using exclusion as a disciplinary sanction [2]. Permanent school exclusion has been rising in the UK since 2014 and the number of permanent exclusions in England in 2016 rose from 6,685 to 7,720 pupils in 2017 [3] and it is particularly prevalent in the age group involved in this project. While some argue that outdoor learning is often evangelised as a panacea for the shortcomings of school environments, particularly for very young learners [4], we draw on the work presented in [5] to make a case for the ways in which outdoor experiences can contribute to the learning needs of older learners at risk of permanent exclusion. We analysed a sample of 102 photographs taken by the project team during the practical sessions in the woodland. We devised a set of categories for coding the images based on our theoretical and pedagogical concerns, and from our reading of empirical literature. Two members of the project team tried out our initial coding categories with the sample in order to check for exhaustiveness and exclusivity, and to try and avoid overlap of codes [6]. Photographs were then coded independently by the four members of the project team using the agreed coding framework. We ask critical questions about the ways in which space, risk, resources, outdoor pedagogies and adult identities can be mobilised to support the learning needs of young people who find school a difficult place to be. In this presentation we will use a selection of photographs to demonstrate that our approach to Visual Content Analysis, drawing on [6] in using a methodologically explicit approach to analysing visual evidence, can produce results that are valid and theoretically ‘interesting’. We interpret the implications of our analysis for educational professionals who want to learn more about preventing permanent exclusion

Individual variation in Achilles tendon morphology and geometry changes susceptibility to injury

The unique structure of the Achilles tendon, combining three smaller sub-tendons, enhances movement efficiency by allowing individual control from connected muscles. This requires compliant interfaces between sub-tendons, but compliance decreases with age and may account for increased injury frequency. Current understanding of sub-tendon sliding and its role in the whole Achilles tendon function is limited. Here we show changing the degree of sliding greatly affects the tendon mechanical behaviour. Our in vitro testing discovered distinct sub-tendon mechanical properties in keeping with their mechanical demands. In silico study based on measured properties, subject-specific tendon geometry, and modified sliding capacity demonstrated age-related displacement reduction similar to our in vivo ultrasonography measurements. Peak stress magnitude and distribution within the whole Achilles tendon are affected by individual tendon geometries, the sliding capacity between sub-tendons, and different muscle loading conditions. These results suggest clinical possibilities to identify patients at risk and design personalised rehabilitation protocols

Antibody-Dependent Enhancement (ADE) of influenza infection and its possible role in the pathogenesis of influenza

Poster Presentations: P05postprintThe International Symposium on 'Surveillance and Discovery in Respiratory and Other Emerging Infections', Phnom Penh, Cambodia, 29-31 May 2011

Antibody-Dependent Enhancement (ADE) of infection and its possible role in the pathogenesis of influenza

Poster Presentationpublished_or_final_versionAnnual Scientific Meeting of the Institut Pasteur International Network, Hong Kong, China, 22-23 November 2010. In BMC Proceedings, 2011, v. 5, suppl. 1, p. 6

The wavelet-NARMAX representation : a hybrid model structure combining polynomial models with multiresolution wavelet decompositions

A new hybrid model structure combing polynomial models with multiresolution wavelet decompositions is introduced for nonlinear system identification. Polynomial models play an important role in approximation theory, and have been extensively used in linear and nonlinear system identification. Wavelet decompositions, in which the basis functions have the property of localization in both time and frequency, outperform many other approximation schemes and offer a flexible solution for approximating arbitrary functions. Although wavelet representations can approximate even severe nonlinearities in a given signal very well, the advantage of these representations can be lost when wavelets are used to capture linear or low-order nonlinear behaviour in a signal. In order to sufficiently utilise the global property of polynomials and the local property of wavelet representations simultaneously, in this study polynomial models and wavelet decompositions are combined together in a parallel structure to represent nonlinear input-output systems. As a special form of the NARMAX model, this hybrid model structure will be referred to as the WAvelet-NARMAX model, or simply WANARMAX. Generally, such a WANARMAX representation for an input-output system might involve a large number of basis functions and therefore a great number of model terms. Experience reveals that only a small number of these model terms are significant to the system output. A new fast orthogonal least squares algorithm, called the matching pursuit orthogonal least squares (MPOLS) algorithm, is also introduced in this study to determine which terms should be included in the final model