Query DAGs: A Practical Paradigm for Implementing Belief-Network Inference

We describe a new paradigm for implementing inference in belief networks, which consists of two steps: (1) compiling a belief network into an arithmetic expression called a Query DAG (Q-DAG); and (2) answering queries using a simple evaluation algorithm. Each node of a Q-DAG represents a numeric operation, a number, or a symbol for evidence. Each leaf node of a Q-DAG represents the answer to a network query, that is, the probability of some event of interest. It appears that Q-DAGs can be generated using any of the standard algorithms for exact inference in belief networks (we show how they can be generated using clustering and conditioning algorithms). The time and space complexity of a Q-DAG generation algorithm is no worse than the time complexity of the inference algorithm on which it is based. The complexity of a Q-DAG evaluation algorithm is linear in the size of the Q-DAG, and such inference amounts to a standard evaluation of the arithmetic expression it represents. The intended value of Q-DAGs is in reducing the software and hardware resources required to utilize belief networks in on-line, real-world applications. The proposed framework also facilitates the development of on-line inference on different software and hardware platforms due to the simplicity of the Q-DAG evaluation algorithm. Interestingly enough, Q-DAGs were found to serve other purposes: simple techniques for reducing Q-DAGs tend to subsume relatively complex optimization techniques for belief-network inference, such as network-pruning and computation-caching.Comment: See http://www.jair.org/ for any accompanying file

Modelling of river discharges using neural networks derived from support vector regression

Neural networks are often used to model complex and nonlinear systems, as they can approximate nonlinear systems with arbitrary accuracy and can be trained from data. Amongst the neural networks, Associative Memory Networks (AMNs) are often used, since they are less computation intensive, and yet good generalization results can be obtained. However, this can only be achieved if the structure of the AMNs is suitably chosen. An approach to choose the structure of the AMNs is to use the Support Vectors (SVs) obtained from the Support Vector Machines. The SVs are obtained from a constrained optimization for a given data set and an error bound. For convenience, this class of AMNs is referred to as the Support Vector Neural Networks (SVNNs). In this paper, the modelling of river discharges with rainfall as input using the SVNN is presented, from which the nonlinear dynamic relationship between rainfall and river discharges is obtained. The prediction of river discharges from the SVNN can give early warning of severe river discharges when there are heavy rainfalls.published_or_final_versio

Parallel approach of a Galerkin-based methodology for predicting the compressive strength of the lightweight aggregate concrete

A methodology based on the Galerkin formulation of the finite element method has been analyzed for predicting the compressive strength of the lightweight aggregate concrete using ultrasonic pulse velocity. Due to both the memory requirements and the computational cost of this technique, its parallelization becomes necessary for solving this problem. For this purpose a mixed MPI/OpenMP parallel algorithm has been designed and different approaches and data distributions analyzed. On the other hand, this Galerkin methodology has been compared with multiple linear regression models, regression trees and artificial neural networks. Based on different measures of goodness of fit, the effectiveness of the Galerkin methodology, compared with these statistical techniques for data mining, is shown.This research was supported by the Spanish Ministry of Science, Innovation and Universities Grant RTI2018-098156-B-C54, co-financed by the European Commission (FEDER funds)

An investigation into adaptive power reduction techniques for neural hardware

In light of the growing applicability of Artificial Neural Network (ANN) in the signal processing field [1] and the present thrust of the semiconductor industry towards lowpower SOCs for mobile devices [2], the power consumption of ANN hardware has become a very important implementation issue. Adaptability is a powerful and useful feature of neural networks. All current approaches for low-power ANN hardware techniques are ‘non-adaptive’ with respect to the power consumption of the network (i.e. power-reduction is not an objective of the adaptation/learning process). In the research work presented in this thesis, investigations on possible adaptive power reduction techniques have been carried out, which attempt to exploit the adaptability of neural networks in order to reduce the power consumption. Three separate approaches for such adaptive power reduction are proposed: adaptation of size, adaptation of network weights and adaptation of calculation precision. Initial case studies exhibit promising results with significantpower reduction

Efficient machine learning: models and accelerations

One of the key enablers of the recent unprecedented success of machine learning is the adoption of very large models. Modern machine learning models typically consist of multiple cascaded layers such as deep neural networks, and at least millions to hundreds of millions of parameters (i.e., weights) for the entire model. The larger-scale model tend to enable the extraction of more complex high-level features, and therefore, lead to a significant improvement of the overall accuracy. On the other side, the layered deep structure and large model sizes also demand to increase computational capability and memory requirements. In order to achieve higher scalability, performance, and energy efficiency for deep learning systems, two orthogonal research and development trends have attracted enormous interests. The first trend is the acceleration while the second is the model compression. The underlying goal of these two trends is the high quality of the models to provides accurate predictions. In this thesis, we address these two problems and utilize different computing paradigms to solve real-life deep learning problems. To explore in these two domains, this thesis first presents the cogent confabulation network for sentence completion problem. We use Chinese language as a case study to describe our exploration of the cogent confabulation based text recognition models. The exploration and optimization of the cogent confabulation based models have been conducted through various comparisons. The optimized network offered a better accuracy performance for the sentence completion. To accelerate the sentence completion problem in a multi-processing system, we propose a parallel framework for the confabulation recall algorithm. The parallel implementation reduce runtime, improve the recall accuracy by breaking the fixed evaluation order and introducing more generalization, and maintain a balanced progress in status update among all neurons. A lexicon scheduling algorithm is presented to further improve the model performance. As deep neural networks have been proven effective to solve many real-life applications, and they are deployed on low-power devices, we then investigated the acceleration for the neural network inference using a hardware-friendly computing paradigm, stochastic computing. It is an approximate computing paradigm which requires small hardware footprint and achieves high energy efficiency. Applying this stochastic computing to deep convolutional neural networks, we design the functional hardware blocks and optimize them jointly to minimize the accuracy loss due to the approximation. The synthesis results show that the proposed design achieves the remarkable low hardware cost and power/energy consumption. Modern neural networks usually imply a huge amount of parameters which cannot be fit into embedded devices. Compression of the deep learning models together with acceleration attracts our attention. We introduce the structured matrices based neural network to address this problem. Circulant matrix is one of the structured matrices, where a matrix can be represented using a single vector, so that the matrix is compressed. We further investigate a more flexible structure based on circulant matrix, called block-circulant matrix. It partitions a matrix into several smaller blocks and makes each submatrix is circulant. The compression ratio is controllable. With the help of Fourier Transform based equivalent computation, the inference of the deep neural network can be accelerated energy efficiently on the FPGAs. We also offer the optimization for the training algorithm for block circulant matrices based neural networks to obtain a high accuracy after compression

Biologically inspired evolutionary temporal neural circuits

Biological neural networks have always motivated creation of new artificial neural networks, and in this case a new autonomous temporal neural network system. Among the more challenging problems of temporal neural networks are the design and incorporation of short and long-term memories as well as the choice of network topology and training mechanism. In general, delayed copies of network signals can form short-term memory (STM), providing a limited temporal history of events similar to FIR filters, whereas the synaptic connection strengths as well as delayed feedback loops (ER circuits) can constitute longer-term memories (LTM). This dissertation introduces a new general evolutionary temporal neural network framework (GETnet) through automatic design of arbitrary neural networks with STM and LTM. GETnet is a step towards realization of general intelligent systems that need minimum or no human intervention and can be applied to a broad range of problems. GETnet utilizes nonlinear moving average/autoregressive nodes and sub-circuits that are trained by enhanced gradient descent and evolutionary search in terms of architecture, synaptic delay, and synaptic weight spaces. The mixture of Lamarckian and Darwinian evolutionary mechanisms facilitates the Baldwin effect and speeds up the hybrid training. The ability to evolve arbitrary adaptive time-delay connections enables GETnet to find novel answers to many classification and system identification tasks expressed in the general form of desired multidimensional input and output signals. Simulations using Mackey-Glass chaotic time series and fingerprint perspiration-induced temporal variations are given to demonstrate the above stated capabilities of GETnet

Sistem Pakar Pengklasifikasi Stadium Kanker Serviks Berbasis Mobile Menggunakan Metode Decision Tree

World Cancer Observation states that in Indonesia cervical cancer ranks second with a total of 9.2% of all cancer cases, which is cervical cancer which continues to increase in percentage every year due to the addition of new cases. It is important for the public to be aware of the symptoms that arise from cervical cancer. Lack of knowledge about cervical cancer from an early age increases the risk of death. This is because the patient knows cervical cancer when it is at an advanced stage. So it is important to know the symptoms of cervical cancer sufferers and their stage level in order to get the appropriate treatment. This study aims to create an expert system that can help the public know the classification of cervical cancer stages. The method used to develop an expert system is the Decision Tree method. One of the decision analysis techniques and classification methods in data mining. The classification process uses 200 records of cervical cancer patients with 12 symptoms as a reference. The Decision Tree method used has an accuracy value of 85.50%, recall 85.40%, and precision 86.74%. The expert system was developed using the flutter framework. The results of the study are in the form of an expert system mobile application that has been black box tested which is declared valid. This system can help the public know the diagnosis results from the symptoms experienced and the stage level accurately to apply the appropriate treatment.Observasi Kanker Dunia menyatakan di Indonesia kanker serviks menempati urutan nomor dua dengan total seluruh kasus kanker 9,2% merupakan kanker serivks yang terus meningkat presentase setiap tahunnya karena  penambahan kasus baru. Penting bagi masyarakat waspada terhadap gejala-gejala yang timbul akibat kanker serviks. Kurangnya pengetahuan tentang kanker serviks  sejak dini meningkatkan risiko kematian. Hal ini disebabkan karena penderita mengetahui kanker serviks ketika berada pada stadium lanjut. Maka penting untuk mengetahui gejala penderita kanker serviks dan tingkat stadiumnya agar mendapatkan penanganan yang sesuai. Penelitian ini bertujuan untuk membuat sistem pakar yang dapat membantu masyarakat mengetahui klasifikasi penyakit stadium kanker serviks. Metode yang digunakan untuk mengembangkan sistem pakar adalah metode decision tree. Salah satu dari teknik analisis keputusan dan metode klasifikasi pada data mining. Proses klasifikasi menggunakan data pasien kanker serviks sebanyak 200 record dengan 12 gejala yang digunakan sebagai acuan.  Metode decision tree yang digunakan memiliki nilai accuracy 85.50%, recall 85.40%, dan precision 86.74%. Sistem pakar  dikembangkan dengan menggunakan frame work flutter. Hasil dari penelitian berupa aplikasi mobile sistem pakar yang telah diuji black box yang dinyatakan valid. Sistem ini dapat membantu masyarakat mengetahui hasil diagnosa dari gejala yang dialami dan tingkat stadium dengan akurat untuk menerapkan penanganan yang sesua

Engineering Algorithms for Route Planning in Multimodal Transportation Networks

Practical algorithms for route planning in transportation networks are a showpiece of successful Algorithm Engineering. This has produced many speedup techniques, varying in preprocessing time, space, query performance, simplicity, and ease of implementation. This thesis explores solutions to more realistic scenarios, taking into account, e.g., traffic, user preferences, public transit schedules, and the options offered by the many modalities of modern transportation networks