Automatic design of deep neural network architectures with evolutionary computation

Deep Neural Networks (DNNs) are algorithms with widespread use in the extraction of knowledge from raw data. DNNs are used to solve problems in the fields of computer vision, natural language understanding, signal processing, and others. DNNs are state-of-the-art machine learning models capable of achieving better results than humans in many tasks. However, their application in fields outside computer science and engineering has been hindered due to the tedious process of trial and error multiple computationally intensive models. Thus, the development of algorithms that could allow for the automatic development of DNNs would further advance the field. Two central problems need to be addressed to allow the automatic design of DNN models: generation and pruning. The automatic generation of DNN architectures would allow for the creation of state-of-the-art models without relying on knowledge from human experts. In contrast, the automatic pruning of DNN architectures would reduce the computational complexity of such models for use in less powerful hardware. The generation and pruning of DNN models can be seen as a combinatorial optimization problem, which can be solved with the tools from the Evolutionary Computation (EC) field. This Ph.D. work proposes the use of Particle Swarm Optimization (PSO) for DNN architecture searching with competitive results and fast convergence, called psoCNN. Another algorithm based on Evolution Strategy (ES) is used for the pruning of DNN architectures, called DeepPruningES. The proposed psoCNN algorithm is capable of finding CNN architectures, a particular type of DNN, for image classification tasks with comparable results to human-crafted DNN models. Likewise, the DeepPruningES algorithm is capable of reducing the number of floating operations of a given DNN model up to 80 percent, and it uses the principles of Multi-Criteria Decision Making (MCDM) to output three pruned model with different trade-offs between computational complexity and classification accuracy. These ideas are then applied to the creation of a unified framework for searching highly accurate, and compact DNN applied for Medical Imaging Diagnostics, and the pruning of Generative Adversarial Networks (GANs) for Medical Imaging Synthesis with competitive results

Study and implementation of neural networks and genetic algorithms to solve the inverse kinematics of a 5-DOF robotic manipulator

Orientador: Marconi Kolm MadridDissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Elétrica e de ComputaçãoResumo: No presente trabalho é mostrado o estudo e a implementação das Redes Neurais Artificiais, RNA, e Algoritmos Genéticos, AG, para resolução da cinemática inversa de um manipulador robótico com 5 graus de liberdade. Todo manipulador robótico é construído com o objetivo de se realize uma determinada tarefa. Para alcançar esse objetivo é necessário o estudo e o emprego dos seus modelos cinemáticos. A descrição matemática do movimento espacial realizado por cada elo do robô é conhecida como Cinemática que é o estudo do movimento de um corpo ou um sistema de corpos rígidos em relação a um referencial cartesiano fixo ignorando as forças e os momentos que causam tal movimento. Existem dois problemas ao se estudar a cinemática de um robô: o problema da cinemática direta e o problema da cinemática inversa. A cinemática inversa é um ramo com grandes desafios devido as equações não serem lineares, dificultando a determinação de soluções de uma forma fechada. Portanto, diversos pesquisadores, ao longo dos anos, tentam resolver esse problema evitando o uso de inversões de equações. Nesse sentido, o uso das redes neurais artificiais e dos algoritmos genéticos se mostram alternativas atraentes. As soluções encontradas no presente trabalho foram aplicadas a um robô educacional com 5 graus de liberdade composto de seis servomotores controlado por um microcontrolador Arduino Uno. O software MATLAB foi utilizado como ferramenta para o desenvolvimento e a aplicação desses dois métodosAbstract: The present work shows the study and implementation of Artificial Neural Networks, ANN, and Genetic Algorithms, AG, to solve the inverse kinematics of a robotic manipulator with 5 degree of freedom. Every robotic manipulator is constructed with the goal of perform a specific task. To reach this goal, the robot needs to track a path, and for that it is necessary the study of its kinematics. The math description of the spatial movement performed by its links is known as kinematics that is the study of the movement of a rigid body or system of rigid bodies in relation to a fixed cartesian reference disregarding the forces and momentums that cause the movement. There are two problems when studying the kinematics: the forward kinematics problem and the inverse kinematics problem. The inverse kinematics is a field of study with challenges due the fact that the equations are not linear which become a problem to obtain closed form solutions. Therefore, many scientists try to solve this problem with methods that do not use equation inversions. In this sense, the use of artificial neural networks and genetic algorithms prove to be interesting alternatives for this purpose. The solutions found in this work were applied to an educational robot platform with 5 degree of freedom and six servomotors controlled by an Arduino Uno microcontroller. The MATLAB software was used as a tool to develop and application of these two methodsMestradoAutomaçãoMestre em Engenharia Elétric

Memory-Based Pruning of Deep Neural Networks for IoT Devices Applied to Flood Detection

Automatic flood detection may be an important component for triggering damage control systems and minimizing the risk of social or economic impacts caused by flooding. Riverside images from regular cameras are a widely available resource that can be used for tackling this problem. Nevertheless, state-of-the-art neural networks, the most suitable approach for this type of computer vision task, are usually resource-consuming, which poses a challenge for deploying these models within low-capability Internet of Things (IoT) devices with unstable internet connections. In this work, we propose a deep neural network (DNN) architecture pruning algorithm capable of finding a pruned version of a given DNN within a user-specified memory footprint. Our results demonstrate that our proposed algorithm can find a pruned DNN model with the specified memory footprint with little to no degradation of its segmentation performance. Finally, we show that our algorithm can be used in a memory-constraint wireless sensor network (WSN) employed to detect flooding events of urban rivers, and the resulting pruned models have competitive results compared with the original models