Mitigating Architectural Mismatch During the Evolutionary Synthesis of Deep Neural Networks

Evolutionary deep intelligence has recently shown great promise for producing small, powerful deep neural network models via the organic synthesis of increasingly efficient architectures over successive generations. Existing evolutionary synthesis processes, however, have allowed the mating of parent networks independent of architectural alignment, resulting in a mismatch of network structures. We present a preliminary study into the effects of architectural alignment during evolutionary synthesis using a gene tagging system. Surprisingly, the network architectures synthesized using the gene tagging approach resulted in slower decreases in performance accuracy and storage size; however, the resultant networks were comparable in size and performance accuracy to the non-gene tagging networks. Furthermore, we speculate that there is a noticeable decrease in network variability for networks synthesized with gene tagging, indicating that enforcing a like-with-like mating policy potentially restricts the exploration of the search space of possible network architectures.Comment: 5 page

Fast YOLO: A Fast You Only Look Once System for Real-time Embedded Object Detection in Video

Object detection is considered one of the most challenging problems in this field of computer vision, as it involves the combination of object classification and object localization within a scene. Recently, deep neural networks (DNNs) have been demonstrated to achieve superior object detection performance compared to other approaches, with YOLOv2 (an improved You Only Look Once model) being one of the state-of-the-art in DNN-based object detection methods in terms of both speed and accuracy. Although YOLOv2 can achieve real-time performance on a powerful GPU, it still remains very challenging for leveraging this approach for real-time object detection in video on embedded computing devices with limited computational power and limited memory. In this paper, we propose a new framework called Fast YOLO, a fast You Only Look Once framework which accelerates YOLOv2 to be able to perform object detection in video on embedded devices in a real-time manner. First, we leverage the evolutionary deep intelligence framework to evolve the YOLOv2 network architecture and produce an optimized architecture (referred to as O-YOLOv2 here) that has 2.8X fewer parameters with just a ~2% IOU drop. To further reduce power consumption on embedded devices while maintaining performance, a motion-adaptive inference method is introduced into the proposed Fast YOLO framework to reduce the frequency of deep inference with O-YOLOv2 based on temporal motion characteristics. Experimental results show that the proposed Fast YOLO framework can reduce the number of deep inferences by an average of 38.13%, and an average speedup of ~3.3X for objection detection in video compared to the original YOLOv2, leading Fast YOLO to run an average of ~18FPS on a Nvidia Jetson TX1 embedded system

Mitigating Architectural Mismatch During the Evolutionary Synthesis of Deep Neural Networks

Evolutionary deep intelligence has recently shown great promisefor producing small, powerful deep neural network models via theorganic synthesis of increasingly efficient architectures over suc-cessive generations. Existing evolutionary synthesis processes,however, have allowed the mating of parent networks independentof architectural alignment, resulting in a mismatch of network struc-tures. We present a preliminary study into the effects of architec-tural alignment during evolutionary synthesis using a gene taggingsystem. Surprisingly, the network architectures synthesized usingthe gene tagging approach resulted in slower decreases in perfor-mance accuracy and storage size; however, the resultant networkswere comparable in size and performance accuracy to the non-gene tagging networks. Furthermore, we speculate that there is anoticeable decrease in network variability for networks synthesizedwith gene tagging, indicating that enforcing a like-with-like matingpolicy potentially restricts the exploration of the search space ofpossible network architectures

Polyploidism in Deep Neural Networks: m-Parent Evolutionary Synthesis of Deep Neural Networks in Varying Population Sizes

Evolutionary deep intelligence was recently proposed to organicallyproduce highly efficient deep neural network architecturesover successive generations. Thus far, current evolutionary synthesisprocesses are based on asexual reproduction, i.e., offspringneural networks are synthesized stochastically from a single parentnetwork. In this study, we investigate the effects of m-parentsexual evolutionary synthesis (m = 1, 2, 3, 5) in combination withvarying population sizes of three, five, and eight synthesized networksper generation. Experimental results were obtained usinga 10% subset of the MNIST handwritten digits dataset, and showthat increasing the number of parent networks results in improvedarchitectural efficiency of the synthesized networks (approximately150x synaptic efficiency and approximately 42–49x cluster efficiency)while resulting in only a 2–3% drop in testing accuracy

Efficient Deep Network Architecture for Vision-Based Vehicle Detection Keyvan Kasiri,

With the progress in intelligent transportation systems in smartcities, vision-based vehicle detection is becoming an important issuein the vision-based surveillance systems. With the advent ofthe big data era, deep learning methods have been increasinglyemployed in the detection, classification, and recognition applicationsdue to their performance accuracy, however, there are stillmajor concerns regarding deployment of such methods in embeddedapplications. This paper offers an efficient process leveragingthe idea of evolutionary deep intelligence on a state-of-the-art deepneural network. Using this approach, the deep neural network isevolved towards a highly sparse set of synaptic weights and clusters.Experimental results for the task of vehicle detection demonstratethat the evolved deep neural network can achieve a substantialimprovement in architecture efficiency adapting for GPUacceleratedapplications without significant sacrifices in detectionaccuracy. The architectural efficiency of ~4X-fold and ~2X-folddecrease is obtained in synaptic weights and clusters, respectively,while the accuracy of 92.8% (drop of less than 4% compared to theoriginal network model) is achieved. Detection results and networkefficiency for the vehicular application are promising, and opensthe door to a wider range of applications in deep learning