852 research outputs found
Design and Evaluation of a Hardware System for Online Signal Processing within Mobile Brain-Computer Interfaces
Brain-Computer Interfaces (BCIs) sind innovative Systeme, die eine direkte Kommunikation zwischen dem Gehirn und externen Geräten ermöglichen. Diese Schnittstellen haben sich zu einer transformativen Lösung nicht nur für Menschen mit neurologischen Verletzungen entwickelt, sondern auch für ein breiteres Spektrum von Menschen, das sowohl medizinische als auch nicht-medizinische Anwendungen umfasst. In der Vergangenheit hat die Herausforderung, dass neurologische Verletzungen nach einer anfänglichen Erholungsphase statisch bleiben, die Forscher dazu veranlasst, innovative Wege zu beschreiten. Seit den 1970er Jahren stehen BCIs an vorderster Front dieser Bemühungen. Mit den Fortschritten in der Forschung haben sich die BCI-Anwendungen erweitert und zeigen ein großes Potenzial für eine Vielzahl von Anwendungen, auch für weniger stark eingeschränkte (zum Beispiel im Kontext von Hörelektronik) sowie völlig gesunde Menschen (zum Beispiel in der Unterhaltungsindustrie). Die Zukunft der BCI-Forschung hängt jedoch auch von der Verfügbarkeit zuverlässiger BCI-Hardware ab, die den Einsatz in der realen Welt gewährleistet.
Das im Rahmen dieser Arbeit konzipierte und implementierte CereBridge-System stellt einen bedeutenden Fortschritt in der Brain-Computer-Interface-Technologie dar, da es die gesamte Hardware zur Erfassung und Verarbeitung von EEG-Signalen in ein mobiles System integriert. Die Architektur der Verarbeitungshardware basiert auf einem FPGA mit einem ARM Cortex-M3 innerhalb eines heterogenen ICs, was Flexibilität und Effizienz bei der EEG-Signalverarbeitung gewährleistet. Der modulare Aufbau des Systems, bestehend aus drei einzelnen Boards, gewährleistet die Anpassbarkeit an unterschiedliche Anforderungen. Das komplette System wird an der Kopfhaut befestigt, kann autonom arbeiten, benötigt keine externe Interaktion und wiegt einschließlich der 16-Kanal-EEG-Sensoren nur ca. 56 g. Der Fokus liegt auf voller Mobilität.
Das vorgeschlagene anpassbare Datenflusskonzept erleichtert die Untersuchung und nahtlose Integration von Algorithmen und erhöht die Flexibilität des Systems. Dies wird auch durch die Möglichkeit unterstrichen, verschiedene Algorithmen auf EEG-Daten anzuwenden, um unterschiedliche Anwendungsziele zu erreichen. High-Level Synthesis (HLS) wurde verwendet, um die Algorithmen auf das FPGA zu portieren, was den Algorithmenentwicklungsprozess beschleunigt und eine schnelle Implementierung von Algorithmusvarianten ermöglicht. Evaluierungen haben gezeigt, dass das CereBridge-System in der Lage ist, die gesamte Signalverarbeitungskette zu integrieren, die für verschiedene BCI-Anwendungen erforderlich ist. Darüber hinaus kann es mit einer Batterie von mehr als 31 Stunden Dauerbetrieb betrieben werden, was es zu einer praktikablen Lösung für mobile Langzeit-EEG-Aufzeichnungen und reale BCI-Studien macht.
Im Vergleich zu bestehenden Forschungsplattformen bietet das CereBridge-System eine bisher unerreichte Leistungsfähigkeit und Ausstattung für ein mobiles BCI. Es erfüllt nicht nur die relevanten Anforderungen an ein mobiles BCI-System, sondern ebnet auch den Weg für eine schnelle Übertragung von Algorithmen aus dem Labor in reale Anwendungen. Im Wesentlichen liefert diese Arbeit einen umfassenden Entwurf für die Entwicklung und Implementierung eines hochmodernen mobilen EEG-basierten BCI-Systems und setzt damit einen neuen Standard für BCI-Hardware, die in der Praxis eingesetzt werden kann.Brain-Computer Interfaces (BCIs) are innovative systems that enable direct communication between the brain and external devices. These interfaces have emerged as a transformative solution not only for individuals with neurological injuries, but also for a broader range of individuals, encompassing both medical and non-medical applications. Historically, the challenge of neurological injury being static after an initial recovery phase has driven researchers to explore innovative avenues. Since the 1970s, BCIs have been at one forefront of these efforts. As research has progressed, BCI applications have expanded, showing potential in a wide range of applications, including those for less severely disabled (e.g. in the context of hearing aids) and completely healthy individuals (e.g. entertainment industry). However, the future of BCI research also depends on the availability of reliable BCI hardware to ensure real-world application.
The CereBridge system designed and implemented in this work represents a significant leap forward in brain-computer interface technology by integrating all EEG signal acquisition and processing hardware into a mobile system. The processing hardware architecture is centered around an FPGA with an ARM Cortex-M3 within a heterogeneous IC, ensuring flexibility and efficiency in EEG signal processing. The modular design of the system, consisting of three individual boards, ensures adaptability to different requirements. With a focus on full mobility, the complete system is mounted on the scalp, can operate autonomously, requires no external interaction, and weighs approximately 56g, including 16 channel EEG sensors.
The proposed customizable dataflow concept facilitates the exploration and seamless integration of algorithms, increasing the flexibility of the system. This is further underscored by the ability to apply different algorithms to recorded EEG data to meet different application goals. High-Level Synthesis (HLS) was used to port algorithms to the FPGA, accelerating the algorithm development process and facilitating rapid implementation of algorithm variants. Evaluations have shown that the CereBridge system is capable of integrating the complete signal processing chain required for various BCI applications. Furthermore, it can operate continuously for more than 31 hours with a 1800mAh battery, making it a viable solution for long-term mobile EEG recording and real-world BCI studies.
Compared to existing research platforms, the CereBridge system offers unprecedented performance and features for a mobile BCI. It not only meets the relevant requirements for a mobile BCI system, but also paves the way for the rapid transition of algorithms from the laboratory to real-world applications. In essence, this work provides a comprehensive blueprint for the development and implementation of a state-of-the-art mobile EEG-based BCI system, setting a new benchmark in BCI hardware for real-world applicability
A Holistic Analysis of Internet of Things (IoT) Security : Principles, Practices, and New Perspectives
Peer reviewedPublisher PD
A Survey of the Security Challenges and Requirements for IoT Operating Systems
The Internet of Things (IoT) is becoming an integral part of our modern lives
as we converge towards a world surrounded by ubiquitous connectivity. The
inherent complexity presented by the vast IoT ecosystem ends up in an
insufficient understanding of individual system components and their
interactions, leading to numerous security challenges. In order to create a
secure IoT platform from the ground up, there is a need for a unifying
operating system (OS) that can act as a cornerstone regulating the development
of stable and secure solutions. In this paper, we present a classification of
the security challenges stemming from the manifold aspects of IoT development.
We also specify security requirements to direct the secure development of an
unifying IoT OS to resolve many of those ensuing challenges. Survey of several
modern IoT OSs confirm that while the developers of the OSs have taken many
alternative approaches to implement security, we are far from engineering an
adequately secure and unified architecture. More broadly, the study presented
in this paper can help address the growing need for a secure and unified
platform to base IoT development on and assure the safe, secure, and reliable
operation of IoT in critical domains.Comment: 13 pages, 2 figure
ABBY: Automating leakage modeling for side-channels analysis
We introduce ABBY, an open-source side-channel leakage
profiling framework that targets the microarchitectural layer.
Existing solutions to characterize the microarchitectural layer
are device-specific and require extensive manual effort. The
main innovation of ABBY is the collection of data, which can
automatically characterize the microarchitecture of a target
device and has the additional benefit of being scalable.
Using ABBY, we create two sets of data which capture the
interaction of instructions for the ARM CORTEX-M0/M3
architecture. These sets are the first to capture detailed information
on the microarchitectural layer. They can be used
to explore various leakage models suitable for creating sidechannel
leakage simulators. A preliminary evaluation of a
leakage model produced with our dataset of real-world cryptographic
implementations shows performance comparable
to state-of-the-art leakage simulators
DEVELOPMENT OF COST-EFFECTIVE FAULT LOCALIZATION PLATFORM FOR UNDERGROUND CABLE SYSTEM
The mining industry is highly dependent on electricity or other forms of energy converted into electricity. Here, one of the common forms of electricity distribution is through underground cables, but the electricity systems regularly confront contingencies due to cable breakdowns or defects. Depending on fault types, detective tools, and cable position, it may take several hours to days to detect the exact fault location with existing methods and tools, which affects the entire mining operation with financial losses and safety issues. Besides, industrial fault localization platforms are very expensive and seldom tailored for the mining industry. To locate such faults, the online fault localization platform has drawn wide attention but is limited in functionalities and applications.
This research aims to develop a cost-effective double-ended online fault localization platform while proposing solutions for data synchronization and accurate traveling wave arrival time detection problems. At first, extensive market research has been done on available platforms to access those platforms’ functionalities and costings. Then, a hardware setup has been developed by combining appropriate sensors, a data acquisition unit, a computational platform, and other necessary components. Furthermore, to establish communication between remote platforms, a Python-based software program has been developed. Besides, query-based server management has been introduced to handle and manage huge amounts of data. Combining the hardware and software, the overall cost of a single platform is CAD $ 2,850.00, which is at least ten times less than the least expensive market option.
The chosen double-ended traveling wave-based online fault localization method requires accurate synchronized time to properly compute the traveling wave arrival time difference. Thus, coordinated universal time alignment of acquired time series data is needed. Global Positioning System (GPS) based universal time synchronization is one of the popular ways. There are several other techniques, but all of these have some practical difficulties and dependencies. To solve this problem, a cost-effective novel zero-crossing point-based data synchronization approach has been proposed. This approach doesn’t rely on GPS receivers or any other existing methods; rather, the measurement is synchronized by calibrating the zero-crossing points of the sinusoid measurements before the fault. In this way, appropriate synchronization has been realized.
The accuracy of traveling wave-based fault localization highly depends on how accurately its arrival times are detected from ends of cable. Accurate detections are achieved when signals sampling rates are high. Usually, a data acquisition system with a higher sampling rate may address the issue, but it increases cost. On the other hand, available interpolation-based up-sampling techniques have several constraints. Machine learning model can solve this problem if trained by real inputs and outputs with desired sampling rates for different types of faults. In this research, a machine learning-based up-sampling model has been presented, which improves the accuracy of traveling wave arrival time detection. Besides, it is cost-effective and outperforms interpolation techniques.
Therefore, the proposed fault localization platform combines all the above solutions, which makes the platform very advance, reliable, and cost-effective
ColibriES: A Milliwatts RISC-V Based Embedded System Leveraging Neuromorphic and Neural Networks Hardware Accelerators for Low-Latency Closed-loop Control Applications
End-to-end event-based computation has the potential to push the envelope in
latency and energy efficiency for edge AI applications. Unfortunately,
event-based sensors (e.g., DVS cameras) and neuromorphic spike-based processors
(e.g., Loihi) have been designed in a decoupled fashion, thereby missing major
streamlining opportunities. This paper presents ColibriES, the first-ever
neuromorphic hardware embedded system platform with dedicated event-sensor
interfaces and full processing pipelines. ColibriES includes event and frame
interfaces and data processing, aiming at efficient and long-life embedded
systems in edge scenarios. ColibriES is based on the Kraken system-on-chip and
contains a heterogeneous parallel ultra-low power (PULP) processor, frame-based
and event-based camera interfaces, and two hardware accelerators for the
computation of both event-based spiking neural networks and frame-based ternary
convolutional neural networks. This paper explores and accurately evaluates the
performance of event data processing on the example of gesture recognition on
ColibriES, as the first step of full-system evaluation. In our experiments, we
demonstrate a chip energy consumption of 7.7 \si{\milli\joule} and latency of
164.5 \si{\milli\second} of each inference with the DVS Gesture event data set
as an example for closed-loop data processing, showcasing the potential of
ColibriES for battery-powered applications such as wearable devices and UAVs
that require low-latency closed-loop control
Electronic Devices for the Combination of Electrically Controlled Drug Release, Electrostimulation, and Optogenetic Stimulation for Nerve Tissue Regeneration
[ES] La capacidad de las células madre para proliferar formando distintas células especializadas les otorga la potencialidad de servir de base para terapias efectivas para patologías cuyo tratamiento era inimaginable hasta hace apenas dos décadas. Sin embargo, esta capacidad se encuentra mediada por estímulos fisiológicos, químicos, y eléctricos, específicos y complejos, que dificultan su traslación a la rutina clínica. Por ello, las células madre representan un campo de estudio en el que se invierten amplios esfuerzos por parte de la comunidad científica.
En el ámbito de la regeneración nerviosa, para modular su desarrollo y diferenciación el tratamiento farmacológico, la electroestimulación, y la estimulación optogenética son técnicas que están consiguiendo prometedores resultados. Es por ello por lo que en la presente tesis se ha desarrollado un conjunto de sistemas electrónicos para permitir la aplicación combinada de estas técnicas in vitro, con perspectiva a su aplicación in vivo.
Hemos diseñado una novedosa tecnología para la liberación eléctricamente controlada de fármacos. Esta tecnología está basada en nanopartículas de sílice mesoporosa y puertas moleculares de bipiridina-heparina. Las puertas moleculares son electroquímicamente reactivas, y encierran los fármacos en el interior de las nanopartículas, liberándolos ante un estímulo eléctrico. Hemos caracterizado esta tecnología, y la hemos validado mediante la liberación controlada de rodamina en cultivos celulares de HeLa. Para la combinación de liberación controlada de fármacos y electroestimulación hemos desarrollado dispositivos que permiten aplicar los estímulos eléctricos de forma configurable desde una interfaz gráfica de usuario. Además, hemos diseñado un módulo de expansión que permite multiplexar las señales eléctricas a diferentes cultivos celulares.
Además, hemos diseñado un dispositivo de estimulación optogenética. Este tipo de estimulación consiste en la modificación genética de las células para que sean sensibles a la radiación lumínica de determinada longitud de onda. En el ámbito de la regeneración de tejido mediante células precursoras neurales, es de interés poder inducir ondas de calcio, favoreciendo su diferenciación en neuronas y la formación de circuitos sinápticos. El dispositivo diseñado permite obtener imágenes en tiempo real mediante microscopía confocal de las respuestas transitorias de las células al ser irradiadas. El dispositivo se ha validado irradiando neuronas modificadas con luz pulsada de 100 ms. También hemos diseñado un dispositivo electrónico complementario de medida de irradiancia con el doble fin de permitir la calibración del equipo de irradiancia y medir la irradiancia en tiempo real durante los experimentos in vitro.
Los resultados del uso de los bioactuadores en procesos complejos y dinámicos, como la regeneración de tejido nervioso, son limitados en lazo abierto. Uno de los principales aspectos analizados es el desarrollo de biosensores que permitiesen la cuantización de ciertas biomoléculas para ajustar la estimulación suministrada en tiempo real. Por ejemplo, la segregación de serotonina es una respuesta identificada en la elongación de células precursoras neurales, pero hay otras biomoléculas de interés para la implementación de un control en lazo cerrado. Entre las tecnologías en el estado del arte, los biosensores basados en transistores de efecto de campo (FET) funcionalizados con aptámeros son realmente prometedores para esta aplicación. Sin embargo, esta tecnología no permitía la medición simultánea de más de una biomolécula objetivo en un volumen reducido debido a las interferencias entre los distintos FETs, cuyos terminales se encuentran inmersos en la solución. Por ello, hemos desarrollado instrumentación electrónica capaz de medir simultáneamente varios de estos biosensores, y la hemos validado mediante la medición simultánea de pH y la detección preliminar de serotonina y glutamato.[CA] La capacitat de les cèl·lules mare per a proliferar formant diferents cèl·lules especialitzades els atorga la potencialitat de servir de base per a teràpies efectives per a patologies el tractament de les quals era inimaginable fins fa a penes dues dècades. No obstant això, aquesta capacitat es troba mediada per estímuls fisiològics, químics, i elèctrics, específics i complexos, que dificulten la seua translació a la rutina clínica. Per això, les cèl·lules mare representen un camp d'estudi en el qual s'inverteixen amplis esforços per part de la comunitat científica.
En l'àmbit de la regeneració nerviosa, per a modular el seu desenvolupament i diferenciació el tractament farmacològic, l'electroestimulació, i l'estimulació optogenética són tècniques que estan aconseguint prometedors resultats. És per això que en la present tesi s'ha desenvolupat un conjunt de sistemes electrònics per a permetre l'aplicació combinada d'aquestes tècniques in vitro, amb perspectiva a la seua aplicació in vivo.
Hem dissenyat una nova tecnologia per a l'alliberament elèctricament controlat de fàrmacs. Aquesta tecnologia està basada en nanopartícules de sílice mesoporosa i portes moleculars de bipiridina-heparina. Les portes moleculars són electroquímicament reactives, i tanquen els fàrmacs a l'interior de les nanopartícules, alliberant-los davant un estímul elèctric. Hem caracteritzat aquesta tecnologia, i l'hem validada mitjançant l'alliberament controlat de rodamina en cultius cel·lulars de HeLa. Per a la combinació d'alliberament controlat de fàrmacs i electroestimulació hem desenvolupat dispositius que permeten aplicar els estímuls elèctrics de manera configurable des d'una interfície gràfica d'usuari. A més, hem dissenyat un mòdul d'expansió que permet multiplexar els senyals elèctrics a diferents cultius cel·lulars.
A més, hem dissenyat un dispositiu d'estimulació optogenètica. Aquest tipus d'estimulació consisteix en la modificació genètica de les cèl·lules perquè siguen sensibles a la radiació lumínica de determinada longitud d'ona. En l'àmbit de la regeneració de teixit mitjançant cèl·lules precursores neurals, és d'interés poder induir ones de calci, afavorint la seua diferenciació en neurones i la formació de circuits sinàptics. El dispositiu dissenyat permet obtindré imatges en temps real mitjançant microscòpia confocal de les respostes transitòries de les cèl·lules en ser irradiades. El dispositiu s'ha validat irradiant neurones modificades amb llum polsada de 100 ms. També hem dissenyat un dispositiu electrònic complementari de mesura d'irradiància amb el doble fi de permetre el calibratge de l'equip d'irradiància i mesurar la irradiància en temps real durant els experiments in vitro.
Els resultats de l'ús dels bioactuadors en processos complexos i dinàmics, com la regeneració de teixit nerviós, són limitats en llaç obert. Un dels principals aspectes analitzats és el desenvolupament de biosensors que permeteren la quantització de certes biomolècules per a ajustar l'estimulació subministrada en temps real. Per exemple, la segregació de serotonina és una resposta identificada amb l'elongació de les cèl·lules precursores neurals, però hi ha altres biomolècules d'interés per a la implementació d'un control en llaç tancat. Entre les tecnologies en l'estat de l'art, els biosensors basats en transistors d'efecte de camp (FET) funcionalitzats amb aptàmers són realment prometedors per a aquesta aplicació. No obstant això, aquesta tecnologia no permetia el mesurament simultani de més d'una biomolècula objectiu en un volum reduït a causa de les interferències entre els diferents FETs, els terminals dels quals es troben immersos en la solució. Per això, hem desenvolupat instrumentació electrònica capaç de mesurar simultàniament diversos d'aquests biosensors i els hem validat amb mesurament simultani del pH i la detecció preliminar de serotonina i glutamat.[EN] The stem cells' ability to proliferate to form different specialized cells gives them the potential to serve as the basis for effective therapies for pathologies whose treatment was unimaginable until just two decades ago. However, this capacity is mediated by specific and complex physiological, chemical, and electrical stimuli that complicate their translation to clinical routine. For this reason, stem cells represent a field of study in which the scientific community is investing a great deal of effort.
In the field of nerve regeneration, to modulate their development and differentiation, pharmacological treatment, electrostimulation, and optogenetic stimulation are techniques that are achieving promising results. For this reason, we have developed a set of electronic systems to allow the combined application of these techniques in vitro, with a view to their application in vivo.
We have designed a novel technology for the electrically controlled release of drugs. This technology is based on mesoporous silica nanoparticles and bipyridine-heparin molecular gates. The molecular gates are electrochemically reactive and entrap the drugs inside the nanoparticles, releasing them upon electrical stimulus. We have characterized this technology and validated it by controlled release of rhodamine in HeLa cell cultures. For combining electrostimulation and controlled drug release we have developed devices that allow applying the different electrical stimuli in a configurable way from a graphical user interface. In addition, we have designed an expansion module that allows multiplexing electrical signals to different cell cultures.
In addition, we have designed an optogenetic stimulation device. This type of stimulation consists of genetically modifying cells to make them sensitive to light radiation of a specific wavelength. In tissue regeneration using neural precursor cells, it is interesting to be able to induce calcium waves, favoring the cell differentiation into neurons and the formation of synaptic circuits. The designed device enable the obtention of real-time images through confocal microscopy of the transient responses of cells upon irradiation. The device has been validated by irradiating modified neurons with 100 ms pulsed light stimulation. We have also designed a complementary electronic irradiance measurement device to allow calibration of the irradiator equipment and measuring irradiance in real time during in vitro experiments.
The results of using bioactuators in complex and dynamic processes, such as nerve tissue regeneration, are limited in an open loop. One of the main aspects analyzed is the development of biosensors that would allow quantifying of specific biomolecules to adjust the stimulation provided in real time. For instance, serotonin secretion is an identified response of neural precursor cells elongation, among other biomolecules of interest for the implementation of a closed-loop control. Among the state-of-the-art technologies, biosensors based on field effect transistors (FETs) functionalized with aptamers are promising for this application. However, this technology did not allow the simultaneous measurement of more than one target biomolecule in a small volume due to interferences between the different FETs, whose terminals are immersed in the solution. This is why we have developed electronic instrumentation capable of simultaneously measuring several of these biosensors, and we have validated it with the simultaneous pH measurement and the preliminary detection of serotonin and glutamate.Monreal Trigo, J. (2023). Electronic Devices for the Combination of Electrically Controlled Drug Release, Electrostimulation, and Optogenetic Stimulation for Nerve Tissue Regeneration [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/19384
Design of a nurse calling system with real time indoor location capabilities
The main driver of this project has been to design a prove of concept of a device that allows live voice communication between patients and medical staff and the capability to locate in real time patients in an enclosed environment. The author of this project had an initial constraint it was supplied by the project director supplied for this project a DWM1001-DEV Module Development Board which provides accurate positioning thanks to its wireless Real Time Location System. After a study of the problem, a selection of components, both hardware and software were selected. The sound system is composed of a I2S Microphone SPH0645 for real time audio capturing, an I2S Amplifier Breakout board MAX98357A and a generic 8ohms speaker. The patient interface component is a button used to trigger communication. For the development of the software the Espressif IoT Development Framework was used, it provides APIs for the user to program the ESP32. The ESP-IDF was installed on VSCode IDE. For debugging the system, we used a J-Link PRO with the Eclipse IDE. The ESP32 communicates with a python-based server using a Wi-Fi network, the communication is based on the UDP protocol. The result is a prototype that showcase that a final product based on this system is feasible, it presents great autonomy and excellent real time communication features. Further lines of work are described, and the presented system is flexible enough to integrate them
Design Techniques of Parallel Accelerator Architectures for Real-Time Processing of Learning Algorithms
H παρούσα διδακτορική διατριβή έχει ως βασικό αντικείμενο μελέτης τα Συνελικτικά Νευρωνικά Δίκτυα (Convolutional Neural Networks - CNNs) για εφαρμογές υπολογιστικής όρασης (computer vision) και συγκεκριμένα εστιάζει στην εκτέλεση της διαδικασίας της εξαγωγής συμπερασμάτων των CNNs (CNN inference) σε ενσωματωμένους επιταχυντές κατάλληλους για εφαρμογές της υπολογιστικής των παρυφών (edge computing). Ο σκοπός της διατριβής είναι να αντιμετωπίσει τις τρέχουσες προκλήσεις σχετικά με τη βελτιστοποίηση των CNNs προκειμένου αυτά να υλοποιηθούν σε edge computing πλατφόρμες, καθώς και τις προκλήσεις στο πεδίο των τεχνικών σχεδίασης αρχιτεκτονικών επιταχυντών για CNNs. Προς αυτή την κατεύθυνση, η παρούσα διατριβή επικεντρώνεται σε διαφορετικές εφαρμογές βαθιάς μάθησης (deep learning), συμπεριλαμβανομένης της επεξεργασίας εικόνων σε δορυφόρους και της πρόβλεψης ηλιακής ακτινοβολίας από εικόνες. Στις παραπάνω εφαρμογές, η διατριβή συμβάλλει σε τέσσερα διακριτά προβλήματα στα πεδία της βελτιστοποίησης CNNs και της σχεδίασης επιταχυντών CNNs.
Αρχικά, η διατριβή συνεισφέρει στην υπάρχουσα βιβλιογραφία σχετικά με τεχνικές επεξεργασίας εικόνας, βασισμένες στα CNNs, για την εκτίμηση και πρόβλεψη ηλιακής ακτινοβολίας. Στα πλαίσια της διατριβής, προτείνεται μια μέθοδος επεξεργασίας εικόνας η οποία βασίζεται στον ακριβή εντοπισμό του Ήλιου σε εικόνες του ουρανού, χρησιμοποιώντας τις συντεταγμένες του Ήλιου και τις εξισώσεις του fisheye φακού της κάμερας λήψης εικόνων του ουρανού. Όταν η προτεινόμενη μέθοδος εφαρμόζεται σε φωτογραφίες του ουρανού πριν από την επεξεργασία τους από τα CNNs, τα αποτελέσματα από την εκτεταμένη μελέτη που διενεργεί η διατριβή, δείχνουν πως μπορεί να βελτιώσει την ακρίβεια των τιμών ακτινοβολίας που παράγουν τα CNNs σε όλες τις περιπτώσεις και με μικρή μόνο αύξηση στο πλήθος των υπολογισμών των CNNs.
Στη συνέχεια, η διδακτορική διατριβή επικεντρώνεται στην κατάτμηση εικόνων βασισμένη στη βαθιά μάθηση, με στόχο τον εντοπισμό σύννεφων από δορυφορικές εικόνες σε εφαρμογές επεξεργασίας δεδομένων σε δορυφόρους. Πιο συγκεκριμένα, στα πλαίσια της διατριβής προτείνεται μια αρχιτεκτονική μοντέλου CNN περιορισμένων υπολογιστικών απαιτήσεων, βασισμένη στην αρχιτεκτονική U-Net, η οποία στοχεύει σε μια βελτιωμένη αναλογία ανάμεσα στο μέγεθος του μοντέλου και στις επιδόσεις του στη δυαδική κατάτμηση της εικόνας. Το προτεινόμενο μοντέλο εκμεταλλεύεται πλήθος τεχνικών CNNs προκειμένου να μειώσει το πλήθος των παραμέτρων και πράξεων που απαιτείται για την εκτέλεση του μοντέλου, αλλά ταυτόχρονα να πετυχαίνει ικανοποιητική ακρίβεια αποτελεσμάτων. Η διατριβή διενεργεί μια μελέτη ανάμεσα σε CNN μοντέλα της βιβλιογραφίας για εντοπισμό σύννεφων που έχουν αξιολογηθεί στα ίδια δεδομένα με το προτεινόμενο μοντέλο, και έτσι αναδεικνύει τα προτερήματά του.
Επιπλέον, η διδακτορική διατριβή στοχεύει στην αποδοτική υλοποίηση του inference των CNNs επεξεργασίας εικόνας σε ενσωματωμένους επιταχυντές κατάλληλους για εφαρμογές edge computing. Για τον σκοπό αυτό, η διατριβή επιλέγει τα Field-Programmable
Gate Arrays (FPGAs) για την επιτάχυνση των CNNs και συνεισφέρει τις λεπτομέρειες της μεθοδολογίας ανάπτυξης που υιοθετήθηκε και η οποία βασίζεται στο εργαλείο Xilinx Vitis AI. Πέρα από τη μελέτη των δυνατοτήτων του Vitis AI, όπως των προχωρημένων τεχνικών κβάντισης των μοντέλων, η διατριβή παρουσιάζει επιπλέον και μια προσέγγιση επιτάχυνσης για την επιτάχυνση των επιμέρους διεργασιών μιας ολοκληρωμένης εργασίας μηχανικής όρασης η οποία εκμεταλλεύεται τους ετερογενείς πόρους του FPGA. Τα αποτελέσματα χρόνων εκτέλεσης και διεκπεραιωτικότητας (throughput) των CNNs τόσο για τη δυαδική κατάτμηση εικόνων για εντοπισμό σύννεφων όσο και για την εκτίμηση ηλιακής ακτινοβολίας από εικόνες, στο FPGA, αναδεικνύουν τις δυνατότητες επεξεργασίας σε πραγματικό χρόνο του επιταχυντή.
Τέλος, η διδακτορική διατριβή συνεισφέρει τη σχεδίαση ενός συστήματος διεπαφής, υψηλών επιδόσεων και με ανοχή στα σφάλματα, για την αμφίδρομη μεταφορά εικόνων ανάμεσα σε ενσωματωμένους επιταχυντές βαθιάς μάθησης, στα πλαίσια υπολογιστικών αρχιτεκτονικών για επεξεργασία δεδομένων σε δορυφόρους. Το σύστημα διεπαφής αναπτύχθηκε για την επικοινωνία ανάμεσα σε ένα FPGA και τον επιταχυντή Intel Movidius Myriad 2 και η εκτεταμένη διαδικασία επαλήθευσης του συστήματος, τόσο σε εμπορικά διαθέσιμες όσο και σε πρωτότυπες πλατφόρμες, έδειξε πως αυτό μπορεί να επιτύχει μέχρι και 2.4 Gbps αμφίδρομους ρυθμούς μετάδοσης δεδομένων εικόνων.The current doctoral thesis focuses on Convolutional Neural Networks (CNNs) for computer vision applications and particularly on the deployment of the inference process of CNNs to embedded accelerators suitable for edge computing. The objective of the thesis is to address several challenges regarding the optimization techniques of CNNs towards their edge deployment as well as challenges in the field of CNN accelerator architectures design techniques.
In this direction, the thesis focuses on different deep learning applications, including on-board payload data processing as well as solar irradiance forecasting, and makes distinct contributions to four different challenges in the fields of CNN optimization and CNN accelerators design.
First, the thesis contributes to the existing literature regarding image processing techniques and deep learning-based image regression for solar irradiance estimation and forecasting.
It proposes an image processing method which is based on accurate sun localization in sky images and which utilizes the solar angles and the mapping functions of the lens of the sky imager camera. When the proposed method is applied to the sky images before these are processed by the image regression CNNs, the results from the extensive study that the thesis conducts, show that the method can improve the accuracy of the irradiance values that the CNNs produce in all cases by introducing only minimal computational overhead.
Next, the thesis focuses on the task of deep learning-based semantic segmentation in order to enable cloud detection from satellite imagery in on-board payload data processing applications. In particular, the thesis proposes a lightweight CNN model architecture, based on the U-Net architecture, which aims at providing an improved trade-off between model size and binary semantic segmentation performance. The proposed model utilizes several CNN techniques in order to reduce the number of parameters and operations required for the inference but at the same time maintain satisfying performance. The thesis conducts a study among CNN models for cloud detection, which are evaluated on the same test dataset as the proposed model, and thus showcases the advantages of the proposed model.
Then, the thesis targets the efficient porting of the inference process of image processing CNNs to edge-oriented embedded accelerator devices. The thesis opts for CNN acceleration based on Field-Programmable Gate Arrays (FPGAs) and contributes the adopted development flow which utilizes the Xilinx Vitis AI framework. Apart from exploring the capabilities of Vitis AI, including its advanced quantization solutions, the thesis also showcases an acceleration approach for accelerating different processes of a single computer vision task by taking advantage of the heterogeneous resources of the FPGA. The execution time and throughput results of the CNN models, for the tasks of binary semantic segmentation for cloud detection as well as image regression for irradiance estimation, on the FPGA, showcase the real-time processing capabilities of the accelerator.
Finally, the thesis contributes the design details of a bi-directional interfacing system for high-throughput and fault-tolerant image transfers between deep learning embedded accelerators, in the context of on-board payload data processing architectures. The interfacing system is developed for interfacing an FPGA with the Intel Movidius Myriad 2 and the extensive testing campaign based on both commercial as well as prototype hardware platforms, shows that it can achieve a bit-rate of up to 2.4 Gbps duplex image data transfers
- …