Neural networks optimization through genetic algorithm searches: A review

Neural networks and genetic algorithms are the two sophisticated machine learning techniques presently attracting attention from scientists, engineers, and statisticians, among others. They have gained popularity in recent years. This paper presents a state of the art review of the research conducted on the optimization of neural networks through genetic algorithm searches. Optimization is aimed toward deviating from the limitations attributed to neural networks in order to solve complex and challenging problems. We provide an analysis and synthesis of the research published in this area according to the application domain, neural network design issues using genetic algorithms, types of neural networks and optimal values of genetic algorithm operators (population size, crossover rate and mutation rate). This study may provide a proper guide for novice as well as expert researchers in the design of evolutionary neural networks helping them choose suitable values of genetic algorithm operators for applications in a specific problem domain. Further research direction, which has not received much attention from scholars, is unveiled

Acoustic Performance of Exhaust Muffler Based Genetic Algorithms and Artificial Neural Network

 The noise level was one of the important indicators as a measure of the quality and performance of the diesel engine.Exhaust noise in diesel engines machine accounted for an important proportion of installed performance exhaust muffler and it was an effective way to control exhaust noise. This article using orthogonal test program for the muffler structure parameters as input to the sound pressure level and diesel fuel each output artificial neural network (BP network) learning sample. Matlab artificial neural network toolbox to complete the training of the network, and better noise performance and fuel consumption rate performance muffler internal structure parameters combination was obtained through genetic algorithm gifted collaborative validation of artificial neural networks and genetic algorithms to optimize application exhaust muffler design is entirely feasible

Neural network computing using on-chip accelerators

The use of neural networks, machine learning, or artificial intelligence, in its broadest and most controversial sense, has been a tumultuous journey involving three distinct hype cycles and a history dating back to the 1960s. Resurgent, enthusiastic interest in machine learning and its applications bolsters the case for machine learning as a fundamental computational kernel. Furthermore, researchers have demonstrated that machine learning can be utilized as an auxiliary component of applications to enhance or enable new types of computation such as approximate computing or automatic parallelization. In our view, machine learning becomes not the underlying application, but a ubiquitous component of applications. This view necessitates a different approach towards the deployment of machine learning computation that spans not only hardware design of accelerator architectures, but also user and supervisor software to enable the safe, simultaneous use of machine learning accelerator resources. In this dissertation, we propose a multi-transaction model of neural network computation to meet the needs of future machine learning applications. We demonstrate that this model, encompassing a decoupled backend accelerator for inference and learning from hardware and software for managing neural network transactions can be achieved with low overhead and integrated with a modern RISC-V microprocessor. Our extensions span user and supervisor software and data structures and, coupled with our hardware, enable multiple transactions from different address spaces to execute simultaneously, yet safely. Together, our system demonstrates the utility of a multi-transaction model to increase energy efficiency improvements and improve overall accelerator throughput for machine learning applications

How Researchers Use Diagrams in Communicating Neural Network Systems

Neural networks are a prevalent and effective machine learning component, and their application is leading to significant scientific progress in many domains. As the field of neural network systems is fast growing, it is important to understand how advances are communicated. Diagrams are key to this, appearing in almost all papers describing novel systems. This paper reports on a study into the use of neural network system diagrams, through interviews, card sorting, and qualitative feedback structured around ecologically-derived examples. We find high diversity of usage, perception and preference in both creation and interpretation of diagrams, examining this in the context of existing design, information visualisation, and user experience guidelines. Considering the interview data alongside existing guidance, we propose guidelines aiming to improve the way in which neural network system diagrams are constructed.Comment: 19 pages, 6 tables, 3 figure

Automatically Designing CNN Architectures for Medical Image Segmentation

Deep neural network architectures have traditionally been designed and explored with human expertise in a long-lasting trial-and-error process. This process requires huge amount of time, expertise, and resources. To address this tedious problem, we propose a novel algorithm to optimally find hyperparameters of a deep network architecture automatically. We specifically focus on designing neural architectures for medical image segmentation task. Our proposed method is based on a policy gradient reinforcement learning for which the reward function is assigned a segmentation evaluation utility (i.e., dice index). We show the efficacy of the proposed method with its low computational cost in comparison with the state-of-the-art medical image segmentation networks. We also present a new architecture design, a densely connected encoder-decoder CNN, as a strong baseline architecture to apply the proposed hyperparameter search algorithm. We apply the proposed algorithm to each layer of the baseline architectures. As an application, we train the proposed system on cine cardiac MR images from Automated Cardiac Diagnosis Challenge (ACDC) MICCAI 2017. Starting from a baseline segmentation architecture, the resulting network architecture obtains the state-of-the-art results in accuracy without performing any trial-and-error based architecture design approaches or close supervision of the hyperparameters changes.Comment: Accepted to Machine Learning in Medical Imaging (MLMI 2018

Towards out-of-distribution generalization in large-scale astronomical surveys: robust networks learn similar representations

The generalization of machine learning (ML) models to out-of-distribution (OOD) examples remains a key challenge in extracting information from upcoming astronomical surveys. Interpretability approaches are a natural way to gain insights into the OOD generalization problem. We use Centered Kernel Alignment (CKA), a similarity measure metric of neural network representations, to examine the relationship between representation similarity and performance of pre-trained Convolutional Neural Networks (CNNs) on the CAMELS Multifield Dataset. We find that when models are robust to a distribution shift, they produce substantially different representations across their layers on OOD data. However, when they fail to generalize, these representations change less from layer to layer on OOD data. We discuss the potential application of similarity representation in guiding model design, training strategy, and mitigating the OOD problem by incorporating CKA as an inductive bias during training.Comment: Accepted to Machine Learning and the Physical Sciences Workshop, NeurIPS 202