Artificial Neural Network Pruning to Extract Knowledge

Artificial Neural Networks (NN) are widely used for solving complex problems from medical diagnostics to face recognition. Despite notable successes, the main disadvantages of NN are also well known: the risk of overfitting, lack of explainability (inability to extract algorithms from trained NN), and high consumption of computing resources. Determining the appropriate specific NN structure for each problem can help overcome these difficulties: Too poor NN cannot be successfully trained, but too rich NN gives unexplainable results and may have a high chance of overfitting. Reducing precision of NN parameters simplifies the implementation of these NN, saves computing resources, and makes the NN skills more transparent. This paper lists the basic NN simplification problems and controlled pruning procedures to solve these problems. All the described pruning procedures can be implemented in one framework. The developed procedures, in particular, find the optimal structure of NN for each task, measure the influence of each input signal and NN parameter, and provide a detailed verbal description of the algorithms and skills of NN. The described methods are illustrated by a simple example: the generation of explicit algorithms for predicting the results of the US presidential election.Comment: IJCNN 202

COMBINED ARTIFICIAL INTELLIGENCE BEHAVIOUR SYSTEMS IN SERIOUS GAMING

This thesis proposes a novel methodology for creating Artificial Agents with semi-realistic behaviour, with such behaviour defined as overcoming common limitations of mainstream behaviour systems; rapidly switching between actions, ignoring “obvious” event priorities, etc. Behaviour in these Agents is not fully realistic as some limitations remain; Agents have a “perfect” knowledge about the surrounding environment, and an inability to transfer knowledge to other Agents (no communication). The novel methodology is achieved by hybridising existing Artificial Intelligence (AI) behaviour systems. In most artificial agents (Agents) behaviour is created using a single behaviour system, whereas this work combines several systems in a novel way to overcome the limitations of each. A further proposal is the separation of behavioural concerns into behaviour systems that are best suited to their needs, as well as describing a biologically inspired memory system that further aids in the production of semi-realistic behaviour. Current behaviour systems are often inherently limited, and in this work it is shown that by combining systems that are complementary to each other, these limitations can be overcome without the need for a workaround. This work examines in detail Belief Desire Intention systems, as well as Finite State Machines and explores how these methodologies can complement each other when combined appropriately. By combining these systems together a hybrid system is proposed that is both fast to react and simple to maintain by separating behaviours into fast-reaction (instinctual) and slow-reaction (behavioural) behaviours, and assigning these to the most appropriate system. Computational intelligence learning techniques such as Artificial Neural Networks have been intentionally avoided, as these techniques commonly present their data in a “black box” system, whereas this work aims to make knowledge explicitly available to the user. A biologically inspired memory system has further been proposed in order to generate additional behaviours in Artificial Agents, such as behaviour related to forgetfulness. This work explores how humans can quickly recall information while still being able to store millions of pieces of information, and how this can be achieved in an artificial system

From a boreal bog to an abandoned peatland pasture: the effect of agricultural management and abandonment on the greenhouse gas fluxes, carbon balance and radiative forcing of a boreal bog in western Newfoundland, Canada

Undisturbed peatlands generally act as a long-term carbon (C) sink and climate cooling. Agriculturally managed peatlands have been identified as hotspots for C and greenhouse gases (GHGs) emissions. However, the increased magnitude of C and GHGs emissions following agriculture management was found to be significantly variable, dependent on the management intensity, peatland initial conditions, cultivation species, time for plant regeneration and fertilization amount. Moreover, the knowledge of how agricultural management and abandonment affects GHGs fluxes is limited by insufficient direct comparisons of GHGs fluxes between undisturbed peatlands and agriculturally managed ones and failure to consider all three GHGs species. To bridge this gap, I conducted a study measuring the landscape-scale carbon dioxide (CO₂) and methane (CH₄) fluxes by eddy covariance and plot-scale N2O fluxes using static chamber technique in a boreal bog and an adjacent abandoned peatland pasture to determine and compare the controls on the temporal patterns of all GHGs fluxes and the effect of agricultural conversion and abandonment on the GHGs fluxes, the C balance and radiative forcing of a boreal bog in western Newfoundland, Canada. This study showed that the gross primary productivity (GPP) and ecosystem respiration (ER) of the abandoned peatland pasture was significantly higher than the counterparts at the bog. The between-site difference in GPP was mainly related to their different vegetation conditions, and the between-site ER difference was linked to different conditions of water table, substrate availability and autotrophic respiration. Overall, the abandoned peatland pasture was a stronger CO₂ sink than the bog. The abandoned peatland pasture was a smaller CH₄ source than the bog. CH₄ flux showed distinct diel and seasonal patterns at the bog but not at the abandoned peatland pasture. Subsurface soil temperature was the main control on CH₄ flux during the growing season but friction velocity became important in the non-growing season at the bog, while no variable was found to be significantly related to the seasonal variation in CH₄ flux at the abandoned peatland pasture. N₂O flux did not show any significant temporal and spatial pattern and the fluxes were very low at both the bog and abandoned peatland pasture. The C and GHGs balance were mainly determined by the magnitude and direction of CO₂ at the pasture, but the GHGs balance was determined by CH₄ flux at the bog. The abandoned peatland pasture acted as a stronger C and GHGs sink than the bog. Therefore, results in this study suggest that the C sequestration capacity and climate cooling function of agriculturally managed peatlands can become stronger than the undisturbed peatlands after long-term abandonment. Research in the thesis contributes new understanding of how agricultural management and abandonment affects the controls on GHGs fluxes and the C balance and climate regulation of peatlands. The work also shows the controls on C fluxes vary over different time-scales and different periods and pose difficulty for incorporation into ecosystem models

Current trends on knowledge extraction and neural networks

This article was presented in a special session of the ICANN 2005 on Neural Networks and Knowledge Extraction. It presents an overview of the current state of the art on the topic. This is an ongoing area of research which helps to explain how a neural network arrives at a given response. This is an important area of research for neural networks if it to gain a wider acceptance in real world applications