MLP neural network based gas classification system on Zynq SoC

Systems based on Wireless Gas Sensor Networks (WGSN) offer a powerful tool to observe and analyse data in complex environments over long monitoring periods. Since the reliability of sensors is very important in those systems, gas classification is a critical process within the gas safety precautions. A gas classification system has to react fast in order to take essential actions in case of fault detection. This paper proposes a low latency real-time gas classification service system, which uses a Multi-Layer Perceptron (MLP) Artificial Neural Network (ANN) to detect and classify the gas sensor data. An accurate MLP is developed to work with the data set obtained from an array of tin oxide (SnO2) gas sensor, based on convex Micro hotplates (MHP). The overall system acquires the gas sensor data through RFID, and processes the sensor data with the proposed MLP classifier implemented on a System on Chip (SoC) platform from Xilinx. Hardware implementation of the classifier is optimized to achieve very low latency for real-time application. The proposed architecture has been implemented on a ZYNQ SoC using fixed-point format and achieved results have shown that an accuracy of 97.4% has been obtained

“Dust in the wind...”, deep learning application to wind energy time series forecasting

To balance electricity production and demand, it is required to use different prediction techniques extensively. Renewable energy, due to its intermittency, increases the complexity and uncertainty of forecasting, and the resulting accuracy impacts all the different players acting around the electricity systems around the world like generators, distributors, retailers, or consumers. Wind forecasting can be done under two major approaches, using meteorological numerical prediction models or based on pure time series input. Deep learning is appearing as a new method that can be used for wind energy prediction. This work develops several deep learning architectures and shows their performance when applied to wind time series. The models have been tested with the most extensive wind dataset available, the National Renewable Laboratory Wind Toolkit, a dataset with 126,692 wind points in North America. The architectures designed are based on different approaches, Multi-Layer Perceptron Networks (MLP), Convolutional Networks (CNN), and Recurrent Networks (RNN). These deep learning architectures have been tested to obtain predictions in a 12-h ahead horizon, and the accuracy is measured with the coefficient of determination, the R² method. The application of the models to wind sites evenly distributed in the North America geography allows us to infer several conclusions on the relationships between methods, terrain, and forecasting complexity. The results show differences between the models and confirm the superior capabilities on the use of deep learning techniques for wind speed forecasting from wind time series data.Peer ReviewedPostprint (published version

심층신경망 학습 가속기 구조 설계: 뉴런의 성김을 이용한 역전파 가속

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 전기·정보공학부, 2020. 8. 이혁재.Deep neural network has become one of the most important technologies in the various fields in computer science which tried to follow the human sense. In some fields, their performance defeats that of human sense with the help of the deep neural network. Since the fact that general purpose GPU can speed up deep neural network, GPU became the main device used for deep neural network. As the complexity of deep neural network becomes high that deep neural network requires more and more computing resources. However, general-purpose GPU consumes a lot of energy that the needs of specific hardware for deep neural network are rising. And nowadays, the specific hardwares are focusing on inference. With complicated network models, training a model consumes enormous time and energy using conventional devices. So there are increasing needs specific hardwares for DNN training. The dissertation exploits deep neural network training accelerator architecture. The training process of a deep neural network (DNN) consists of three phases: forward propagation, backward propagation, and weight update. Among these, backward propagation for calculating gradients of activations is the most time consuming phase. The dissertation proposes hardware architectures to accelerate DNN training, focus- ing on the backward propagation phase. The dissertation makes use of the sparsity of the neurons incurred by ReLU layer or dropout layer to accelerate the backward propagation. The first part of the dissertation proposes a hardware architecture to accelerate DNN backward propagation for convolutional layer. We assume using rectified linear unit (ReLU), which is the most widely used activation function. Since the output as well as the derivative of ReLU is zero for negative inputs, the gradient for activation is also zero for negative values. Thus, it is not needed to calculate the gradient of input activation if the input activation value is zero. Based on this observation, we design an efficient DNN accelerating hardware that skips the gradient computations for zero activations. We show the effectiveness of the approach through experiments with our accelerator design. The second part of the dissertation proposes a hardware architecture for fully connected layer. Similar to ReLU layer, dropoout layer has explicit zero gradient for the dropped activation without gradient computation. Dropout is one of the regulariza- tion techniques which can solve the overfitting problem. During the DNN training, the dropout disconnect connections between neurons. Since the error does not propagated through the disconnected connections, we can detect zero gradient becomre computation. Making use of this characteristics, the dissertation proposes a hardware which can accelerate the backward propagation of fully connected layer. Further, the dissertation showed the effectiveness of the approach through simulation.심층신경망은 컴퓨터 과학의 다양한 분야 중 인간의 감각을 쫓는 분야에서 가장 중요한 기술이 되어왔다. 몇몇 분야에서는 이미 심층신경망의 도움으로 인간의 감 각을 뛰어넘은 분야도 존재한다. GPGPU를 이용한 심층신경망의 가속이 가능해진 이후, GPU는 심층신경망에 있어 가장 주요한 장치로 사용되고 있다. 심층신경망 의 복잡도가 높아짐에 따라 연산에 더 많은 컴퓨팅 자원을 요구하고 있다. 그러나 GPGPU는 에너지 소모가 크기에 효율적인 심층신경망 전용 하드웨어 개발에 대한 요구가 증가하고 있다. 현재까지 이러한 전용 하드웨어는 주로 심층신경망 추론에 집중되어 왔다. 복잡한 심층신경망 모델은 학습에 긴 시간이 들고 많은 에너지를 소 모한다. 이에 심층신경망 학습을 위한 전용 하드웨어에 대한 요구가 늘어가고 있다. 본 학위논문은 심층신경망 학습 가속기 구조를 탐색하였다. 심층신경망의 학습 은 순전파, 역전파, 가중치 갱신 이렇게 세 단계로 이루어져 있다. 이 중 액티베이 션의 그래디언트를 구하는 역전파 단계가 가장 시간이 오래 걸리는 단계이다. 본 학위논문에서는 역전파 단계에 중점을 둔 심층신경망 학습을 가속하는 하드웨어 구조를 제안한다. ReLU 레이어 혹은 dropout 레이어로 인해 생긴 뉴론의 성김을 이용하여 심층신경망 학습의 역전파를 가속한다. 학위논문의 첫 부분은 합성곱 신경망의 역전파를 가속하는 심층신경망 학습 하 드웨어이다. 가장 많이 쓰이는 활성화 함수인 ReLU를 이용하는 신경망을 가정했다. 음수 입력값에 대한 ReLU 활성화 함수의 도함수가 0이 되어 해당 액티베이션의 그 래디언트 또한 0이 된다. 이 경우 그래디언트 값에 대한 계산 없이도 그래디언트 값이 0이 되는 것을 알 수 있기에 해당 그래디언트는 계산하지 않아도 된다. 이러한 특성을 이용하여 0값인 액티베이션에 대한 그래디언트 계산을 건너 뛸 수 있는 효율적인 심층신경망 가속 하드웨어를 설계했다. 또한 실험을 통해 본 하드웨어의 효율성을 검증했다. 학위논문의 두번째 부분은 완전연결 신경망의 학습을 가속하는 하드웨어 구조 제안이다. ReLU 레이어와 비슷하게 dropout 레이어 또한 그래디언트 계산 없이도 그 결과가 0임을 알 수 있다. Dropout은 심층신경망의 과적합을 해결하는 일반화 기법 중 하나로, 심층신경망 학습 과정 동안에만 무작위로 신경망의 연결을 끊어 놓는다. 신경망이 끊어진 경로로는 역전파 단계에서 에러가 전파되지 않기에 해당 그래디언트 값 또한 0임을 미리 알 수 있다. 이 특성을 이용하여 완전연결 신경망의 역전파를 가속할 수 있는 하드웨어를 설계했다. 또한 시뮬레이션을 통해 본 하드웨 어의 효율성을 검증했다.1 Introduction 1 1.1 Deep Neural Network Training 4 1.2 Convolutional Neural Network 5 1.2.1 Forward propagation 5 1.2.2 Backward propagation 6 1.2.3 Weight update 6 1.3 Rectified Linear Unit 7 1.4 Dropout 8 1.5 Previous Works 9 2 Acceleration of DNN Backward Propagation on CNN layer 12 2.1 Motivation 12 2.2 Selective Gradient Computation for Zero Activations 17 2.2.1 Baseline Architecture 17 2.2.2 Bit-Vector for Selective Gradient Computation 20 2.2.3 Filter Collector 23 2.2.4 Zero-Gradient Insertion in Write DMA 26 2.3 Overall Architecture 27 2.4 SRAM Buffer 28 2.4.1 Motivation 28 2.4.2 Selective Gradient Computation and SRAM Buffer 29 2.5 Experimental Results 32 2.5.1 Performance Simulator 32 2.5.2 RTL Implementation 33 2.5.3 Performance Improvement 35 2.5.4 Energy Reduction 37 2.5.5 Impacts of SRAM Buffer 39 2.6 Summary 45 3 Acceleration of DNN Backward Propagation on Fully Connected Layer 46 3.1 Motivation 46 3.1.1 Dropout 46 3.1.2 Conventional Dropout Layer Implementations 46 3.1.3 Applications of Dropout 47 3.2 Selective Gradient Computation for Dropped Activations 50 3.2.1 Baseline Architecture 50 3.2.2 Filter Dropper 50 3.3 Overall Architecture 54 3.4 Experimental Results 55 3.4.1 Simulator and Benchmark 55 3.4.2 Results 55 3.5 Summary 56 4 Conclusion 57Docto

Proposal of an adaptive infotainment system depending on driving scenario complexity

Tesi en modalitat Doctorat industrialPla de Doctorat industrial de la Generalitat de CatalunyaThe PhD research project is framed within the plan of industrial doctorates of the “Generalitat de Catalunya”. During the investigation, most of the work was carried out at the facilities of the vehicle manufacturer SEAT, specifically at the information and entertainment (infotainment) department. In the same way, there was a continuous cooperation with the telematics department of the UPC. The main objective of the project consisted in the design and validation of an adaptive infotainment system dependent on the driving complexity. The system was created with the purpose of increasing driver’ experience while guaranteeing a proper level of road safety. Given the increasing number of application and services available in current infotainment systems, it becomes necessary to devise a system capable of balancing these two counterparts. The most relevant parameters that can be used for balancing these metrics while driving are: type of services offered, interfaces available for interacting with the services, the complexity of driving and the profile of the driver. The present study can be divided into two main development phases, each phase had as outcome a real physical block that came to be part of the final system. The final system was integrated in a vehicle and validated in real driving conditions. The first phase consisted in the creation of a model capable of estimating the driving complexity based on a set of variables related to driving. The model was built by employing machine learning methods and the dataset necessary to create it was collected from several driving routes carried out by different participants. This phase allowed to create a model capable of estimating, with a satisfactory accuracy, the complexity of the road using easily extractable variables in any modern vehicle. This approach simplify the implementation of this algorithm in current vehicles. The second phase consisted in the classification of a set of principles that allow the design of the adaptive infotainment system based on the complexity of the road. These principles are defined based on previous researches undertaken in the field of usability and user experience of graphical interfaces. According to these of principles, a real adaptive infotainment system with the most commonly used functionalities; navigation, radio and media was designed and integrated in a real vehicle. The developed system was able to adapt the presentation of the content according to the estimation of the driving complexity given by the block developed in phase one. The adaptive system was validated in real driving scenarios by several participants and results showed a high level of acceptance and satisfaction towards this adaptive infotainment. As a starting point for future research, a proof of concept was carried out to integrate new interfaces into a vehicle. The interface used as reference was a Head Mounted screen that offered redundant information in relation to the instrument cluster. Tests with participants served to understand how users perceive the introduction of new technologies and how objective benefits could be blurred by initial biases.El proyecto de investigación de doctorado se enmarca dentro del plan de doctorados industriales de la Generalitat de Catalunya. Durante la investigación, la mayor parte del trabajo se llevó a cabo en las instalaciones del fabricante de vehículos SEAT, específicamente en el departamento de información y entretenimiento (infotainment). Del mismo modo, hubo una cooperación continua con el departamento de telemática de la UPC. El objetivo principal del proyecto consistió en el diseño y la validación de un sistema de información y entretenimiento adaptativo que se ajustaba de acuerdo a la complejidad de la conducción. El sistema fue creado con el propósito de aumentar la experiencia del conductor y garantizar un nivel adecuado en la seguridad vial. El proyecto surge dado el número creciente de aplicaciones y servicios disponibles en los sistemas actuales de información y entretenimiento; es por ello que se hace necesario contar con un sistema capaz de equilibrar estas dos contrapartes. Los parámetros más relevantes que se pueden usar para equilibrar estas métricas durante la conducción son: el tipo de servicios ofrecidos, las interfaces disponibles para interactuar con los servicios, la complejidad de la conducción y el perfil del conductor. El presente estudio se puede dividir en dos fases principales de desarrollo, cada fase tuvo como resultado un componente que se convirtió en parte del sistema final. El sistema final fue integrado en un vehículo y validado en condiciones reales de conducción. La primera fase consistió en la creación de un modelo capaz de estimar la complejidad de la conducción en base a un conjunto de variables relacionadas con la conducción. El modelo se construyó empleando "Machine Learning Methods" y el conjunto de datos necesario para crearlo se recopiló a partir de varias rutas de conducción realizadas por diferentes participantes. Esta fase permitió crear un modelo capaz de estimar, con una precisión satisfactoria, la complejidad de la carretera utilizando variables fácilmente extraíbles en cualquier vehículo moderno. Este enfoque simplifica la implementación de este algoritmo en los vehículos actuales. La segunda fase consistió en la clasificación de un conjunto de principios que permiten el diseño del sistema de información y entretenimiento adaptativo basado en la complejidad de la carretera. Estos principios se definen en base a investigaciones anteriores realizadas en el campo de usabilidad y experiencia del usuario con interfaces gráficas. De acuerdo con estos principios, un sistema de entretenimiento y entretenimiento real integrando las funcionalidades más utilizadas; navegación, radio y audio fue diseñado e integrado en un vehículo real. El sistema desarrollado pudo adaptar la presentación del contenido según la estimación de la complejidad de conducción dada por el bloque desarrollado en la primera fase. El sistema adaptativo fue validado en escenarios de conducción reales por varios participantes y los resultados mostraron un alto nivel de aceptación y satisfacción hacia este entretenimiento informativo adaptativo. Como punto de partida para futuras investigaciones, se llevó a cabo una prueba de concepto para integrar nuevas interfaces en un vehículo. La interfaz utilizada como referencia era una pantalla a la altura de los ojos (Head Mounted Display) que ofrecía información redundante en relación con el grupo de instrumentos. Las pruebas con los participantes sirvieron para comprender cómo perciben los usuarios la introducción de nuevas tecnologías y cómo los sesgos iniciales podrían difuminar los beneficios.Postprint (published version

Proposal of an adaptive infotainment system depending on driving scenario complexity

The PhD research project is framed within the plan of industrial doctorates of the “Generalitat de Catalunya”. During the investigation, most of the work was carried out at the facilities of the vehicle manufacturer SEAT, specifically at the information and entertainment (infotainment) department. In the same way, there was a continuous cooperation with the telematics department of the UPC. The main objective of the project consisted in the design and validation of an adaptive infotainment system dependent on the driving complexity. The system was created with the purpose of increasing driver’ experience while guaranteeing a proper level of road safety. Given the increasing number of application and services available in current infotainment systems, it becomes necessary to devise a system capable of balancing these two counterparts. The most relevant parameters that can be used for balancing these metrics while driving are: type of services offered, interfaces available for interacting with the services, the complexity of driving and the profile of the driver. The present study can be divided into two main development phases, each phase had as outcome a real physical block that came to be part of the final system. The final system was integrated in a vehicle and validated in real driving conditions. The first phase consisted in the creation of a model capable of estimating the driving complexity based on a set of variables related to driving. The model was built by employing machine learning methods and the dataset necessary to create it was collected from several driving routes carried out by different participants. This phase allowed to create a model capable of estimating, with a satisfactory accuracy, the complexity of the road using easily extractable variables in any modern vehicle. This approach simplify the implementation of this algorithm in current vehicles. The second phase consisted in the classification of a set of principles that allow the design of the adaptive infotainment system based on the complexity of the road. These principles are defined based on previous researches undertaken in the field of usability and user experience of graphical interfaces. According to these of principles, a real adaptive infotainment system with the most commonly used functionalities; navigation, radio and media was designed and integrated in a real vehicle. The developed system was able to adapt the presentation of the content according to the estimation of the driving complexity given by the block developed in phase one. The adaptive system was validated in real driving scenarios by several participants and results showed a high level of acceptance and satisfaction towards this adaptive infotainment. As a starting point for future research, a proof of concept was carried out to integrate new interfaces into a vehicle. The interface used as reference was a Head Mounted screen that offered redundant information in relation to the instrument cluster. Tests with participants served to understand how users perceive the introduction of new technologies and how objective benefits could be blurred by initial biases.El proyecto de investigación de doctorado se enmarca dentro del plan de doctorados industriales de la Generalitat de Catalunya. Durante la investigación, la mayor parte del trabajo se llevó a cabo en las instalaciones del fabricante de vehículos SEAT, específicamente en el departamento de información y entretenimiento (infotainment). Del mismo modo, hubo una cooperación continua con el departamento de telemática de la UPC. El objetivo principal del proyecto consistió en el diseño y la validación de un sistema de información y entretenimiento adaptativo que se ajustaba de acuerdo a la complejidad de la conducción. El sistema fue creado con el propósito de aumentar la experiencia del conductor y garantizar un nivel adecuado en la seguridad vial. El proyecto surge dado el número creciente de aplicaciones y servicios disponibles en los sistemas actuales de información y entretenimiento; es por ello que se hace necesario contar con un sistema capaz de equilibrar estas dos contrapartes. Los parámetros más relevantes que se pueden usar para equilibrar estas métricas durante la conducción son: el tipo de servicios ofrecidos, las interfaces disponibles para interactuar con los servicios, la complejidad de la conducción y el perfil del conductor. El presente estudio se puede dividir en dos fases principales de desarrollo, cada fase tuvo como resultado un componente que se convirtió en parte del sistema final. El sistema final fue integrado en un vehículo y validado en condiciones reales de conducción. La primera fase consistió en la creación de un modelo capaz de estimar la complejidad de la conducción en base a un conjunto de variables relacionadas con la conducción. El modelo se construyó empleando "Machine Learning Methods" y el conjunto de datos necesario para crearlo se recopiló a partir de varias rutas de conducción realizadas por diferentes participantes. Esta fase permitió crear un modelo capaz de estimar, con una precisión satisfactoria, la complejidad de la carretera utilizando variables fácilmente extraíbles en cualquier vehículo moderno. Este enfoque simplifica la implementación de este algoritmo en los vehículos actuales. La segunda fase consistió en la clasificación de un conjunto de principios que permiten el diseño del sistema de información y entretenimiento adaptativo basado en la complejidad de la carretera. Estos principios se definen en base a investigaciones anteriores realizadas en el campo de usabilidad y experiencia del usuario con interfaces gráficas. De acuerdo con estos principios, un sistema de entretenimiento y entretenimiento real integrando las funcionalidades más utilizadas; navegación, radio y audio fue diseñado e integrado en un vehículo real. El sistema desarrollado pudo adaptar la presentación del contenido según la estimación de la complejidad de conducción dada por el bloque desarrollado en la primera fase. El sistema adaptativo fue validado en escenarios de conducción reales por varios participantes y los resultados mostraron un alto nivel de aceptación y satisfacción hacia este entretenimiento informativo adaptativo. Como punto de partida para futuras investigaciones, se llevó a cabo una prueba de concepto para integrar nuevas interfaces en un vehículo. La interfaz utilizada como referencia era una pantalla a la altura de los ojos (Head Mounted Display) que ofrecía información redundante en relación con el grupo de instrumentos. Las pruebas con los participantes sirvieron para comprender cómo perciben los usuarios la introducción de nuevas tecnologías y cómo los sesgos iniciales podrían difuminar los beneficios