A 64mW DNN-based Visual Navigation Engine for Autonomous Nano-Drones

Fully-autonomous miniaturized robots (e.g., drones), with artificial intelligence (AI) based visual navigation capabilities are extremely challenging drivers of Internet-of-Things edge intelligence capabilities. Visual navigation based on AI approaches, such as deep neural networks (DNNs) are becoming pervasive for standard-size drones, but are considered out of reach for nanodrones with size of a few cm${}^\mathrm{2}$. In this work, we present the first (to the best of our knowledge) demonstration of a navigation engine for autonomous nano-drones capable of closed-loop end-to-end DNN-based visual navigation. To achieve this goal we developed a complete methodology for parallel execution of complex DNNs directly on-bard of resource-constrained milliwatt-scale nodes. Our system is based on GAP8, a novel parallel ultra-low-power computing platform, and a 27 g commercial, open-source CrazyFlie 2.0 nano-quadrotor. As part of our general methodology we discuss the software mapping techniques that enable the state-of-the-art deep convolutional neural network presented in [1] to be fully executed on-board within a strict 6 fps real-time constraint with no compromise in terms of flight results, while all processing is done with only 64 mW on average. Our navigation engine is flexible and can be used to span a wide performance range: at its peak performance corner it achieves 18 fps while still consuming on average just 3.5% of the power envelope of the deployed nano-aircraft.Comment: 15 pages, 13 figures, 5 tables, 2 listings, accepted for publication in the IEEE Internet of Things Journal (IEEE IOTJ

An FPGA Noise Resistant Digital Temperature Sensor with Auto Calibration

In recent years, thermal sensing in digital devices has become increasingly important. From a security perspective, new thermal-based attacks have revealed vulnerabilities in digital devices. Traditional temperature sensors using analog-to-digital converters consume significant power and are not conducive to rapid development. As a result, there has been an escalating demand for low cost, low power digital temperature sensors that can be seamlessly integrated onto digital devices. This research seeks to create a modular Field Programmable Gate Array digital temperature sensor with auto one-point calibration to eliminate the excessive costs and time associated with calibrating existing digital temperature sensors. In addition, to support the auxiliary protection role, the sensor is evaluated alongside a RSA circuit implemented on the same chip, with methods developed to mitigate noise and power fluctuations introduced by the main circuit. The result is a digital temperature sensor resistant to noise and suitable for quick mass deployment in digital devices

Adaptive heterogeneous parallelism for semi-empirical lattice dynamics in computational materials science.

With the variability in performance of the multitude of parallel environments available today, the conceptual overhead created by the need to anticipate runtime information to make design-time decisions has become overwhelming. Performance-critical applications and libraries carry implicit assumptions based on incidental metrics that are not portable to emerging computational platforms or even alternative contemporary architectures. Furthermore, the significance of runtime concerns such as makespan, energy efficiency and fault tolerance depends on the situational context. This thesis presents a case study in the application of both Mattsons prescriptive pattern-oriented approach and the more principled structured parallelism formalism to the computational simulation of inelastic neutron scattering spectra on hybrid CPU/GPU platforms. The original ad hoc implementation as well as new patternbased and structured implementations are evaluated for relative performance and scalability. Two new structural abstractions are introduced to facilitate adaptation by lazy optimisation and runtime feedback. A deferred-choice abstraction represents a unified space of alternative structural program variants, allowing static adaptation through model-specific exhaustive calibration with regards to the extrafunctional concerns of runtime, average instantaneous power and total energy usage. Instrumented queues serve as mechanism for structural composition and provide a representation of extrafunctional state that allows realisation of a market-based decentralised coordination heuristic for competitive resource allocation and the Lyapunov drift algorithm for cooperative scheduling

Advances in analytical models and applications for RFID, WSN and AmI systems

Experimentos llevados a cabo con el equipo de división de honor UCAM Volleyball Murcia.[SPA] Internet de las cosas (IoT) integra distintos elementos que actúan tanto como fuentes, como sumideros de información, a diferencia de la percepción que se ha tenido hasta ahora de Internet, centrado en las personas. Los avances en IoT engloban un amplio número de áreas y tecnologías, desde la adquisición de información hasta el desarrollo de nuevos protocolos y aplicaciones. Un concepto clave que subyace en el concepto de IoT, es el procesamiento de forma inteligente y autónoma de los flujos de información que se dispone. En este trabajo, estudiamos tres aspectos diferentes de IoT. En primer lugar, nos centraremos en la infraestructura de obtención de datos. Entre las diferentes tecnologías de obtención de datos disponibles en los sistemas IoT, la Identificación por Radio Frecuencia (RFID) es considerada como una de las tecnologías predominantes. RFID es la tecnología detrás de aplicaciones tales como control de acceso, seguimiento y rastreo de contenedores, gestión de archivos, clasificación de equipaje o localización de equipos. Con el auge de la tecnología RFID, muchas instalaciones empiezan a requerir la presencia de múltiples lectores RFID que operan próximos entre sí y conjuntamente. A estos escenarios se les conoce como dense reader environments (DREs). La coexistencia de varios lectores operando simultáneamente puede causar graves problemas de interferencias en el proceso de identificación. Uno de los aspectos claves a resolver en los RFID DREs consiste en lograr la coordinación entre los lectores. Estos problemas de coordinación son tratados en detalle en esta tesis doctoral. Además, dentro del área de obtención de datos relativa a IoT, las Redes de Sensores Inalámbricas (WSNs) desempeñan un papel fundamental. Durante la última década, las WSNs han sido estudiadas ampliamente de forma teórica, y la mayoría de problemas relacionados con la comunicación en este tipo de redes se han conseguido resolver de forma favorable. Sin embargo, con la implementación de WSNs en proyectos reales, han surgido nuevos problemas, siendo uno de ellos el desarrollo de estrategias realistas para desplegar las WSN. En este trabajo se estudian diferentes métodos que resuelven este problema, centrándonos en distintos criterios de optimización, y analizando las diferentes ventajas e inconvenientes que se producen al buscar una solución equilibrada. Por último, la Inteligencia Ambiental (AmI) forma parte del desarrollo de aplicaciones inteligentes en IoT. Hasta ahora, han sido las personas quienes han tenido que adaptarse al entorno, en cambio, AmI persigue crear entornos de obtención de datos capaces de anticipar y apoyar las acciones de las personas. AmI se está introduciendo progresivamente en diversos entornos reales tales como el sector de la educación y la salud, en viviendas, etc. En esta tesis se introduce un sistema AmI orientado al deporte que busca mejorar el entrenamiento de los atletas, siendo el objetivo prioritario el desarrollo de un asistente capaz de proporcionar órdenes de entrenamiento, basadas tanto en el entorno como en el rendimiento de los atletas. [ENG] Internet of Things (IoT) is being built upon many different elements acting as sources and sinks of information, rather than the previous human-centric Internet conception. Developments in IoT include a vast set of fields ranging from data sensing, to development of new protocols and applications. Indeed, a key concept underlying in the conception of IoT is the smart and autonomous processing of the new huge data flows available. In this work, we aim to study three different aspects within IoT. First, we will focus on the sensing infrastructure. Among the different kind of sensing technologies available to IoT systems, Radio Frequency Identification (RFID) is widely considered one of the leading technologies. RFID is the enabling technology behind applications such as access control, tracking and tracing of containers, file management, baggage sorting or equipment location. With the grow up of RFID, many facilities require multiple RFID readers usually operating close to each other. These are known as Dense Reader Environments (DREs). The co-existence of several readers operating concurrently is known to cause severe interferences on the identification process. One of the key aspects to solve in RFID DREs is achieving proper coordination among readers. This is the focus of the first part of this doctoral thesis. Unlike previous works based on heuristics, we address this problem through an optimization-based approach. The goal is identifying the maximum mean number of tags while network constraints are met. To be able to formulate these optimization problems, we have obtained analytically the mean number of identifications in a bounded -discrete or continuous- time period, an additional novel contribution of our work. Results show that our approach is overwhelmingly better than previous known methods. Along sensing technologies of IoT, Wireless Sensor Networks (WSNs) plays a fundamental role. WSNs have been largely and theoretically studied in the past decade, and many of their initial problems related to communication aspects have been successfully solved. However, with the adoption of WSNs in real-life projects, new issues have arisen, being one of them the development of realistic strategies to deploy WSNs. We have studied different ways of solving this aspect by focusing on different optimality criteria and evaluating the different trade-offs that occur when a balanced solution must be selected. On the one hand, deterministic placements subject to conflicting goals have been addressed. Results can be obtained in the form of Pareto-frontiers, allowing proper solution selection. On the other hand, a number of situations correspond to deployments were the nodes¿ position is inherently random. We have analyzed these situations leading first to a theoretical model, which later has been particularized to a Moon WSN survey. Our work is the first considering a full model with realistic properties such as 3D topography, propellant consumptions or network lifetime and mass limitations. Furthermore, development of smart applications within IoT is the focus of the Ambient Intelligence (AmI) field. Rather than having people adapting to the surrounding environment, AmI pursues the development of sensitive environments able to anticipate support in people¿s actions. AmI is progressively being introduced in many real-life environments like education, homes, health and so forth. In this thesis we develop a sport-oriented AmI system designed to improve athletes training. The goal is developing an assistant able to provide real-time training orders based on both environment and athletes¿ biometry, which is aimed to control the aerobic and the technical-tactical training. Validation experiments with the honor league UCAM Volleyball Murcia team have shown the suitability of this approach.[ENG] Internet of Things (IoT) is being built upon many different elements acting as sources and sinks of information, rather than the previous human-centric Internet conception. Developments in IoT include a vast set of fields ranging from data sensing, to development of new protocols and applications. Indeed, a key concept underlying in the conception of IoT is the smart and autonomous processing of the new huge data flows available. In this work, we aim to study three different aspects within IoT. First, we will focus on the sensing infrastructure. Among the different kind of sensing technologies available to IoT systems, Radio Frequency Identification (RFID) is widely considered one of the leading technologies. RFID is the enabling technology behind applications such as access control, tracking and tracing of containers, file management, baggage sorting or equipment location. With the grow up of RFID, many facilities require multiple RFID readers usually operating close to each other. These are known as Dense Reader Environments (DREs). The co-existence of several readers operating concurrently is known to cause severe interferences on the identification process. One of the key aspects to solve in RFID DREs is achieving proper coordination among readers. This is the focus of the first part of this doctoral thesis. Unlike previous works based on heuristics, we address this problem through an optimization-based approach. The goal is identifying the maximum mean number of tags while network constraints are met. To be able to formulate these optimization problems, we have obtained analytically the mean number of identifications in a bounded -discrete or continuous- time period, an additional novel contribution of our work. Results show that our approach is overwhelmingly better than previous known methods. Along sensing technologies of IoT, Wireless Sensor Networks (WSNs) plays a fundamental role. WSNs have been largely and theoretically studied in the past decade, and many of their initial problems related to communication aspects have been successfully solved. However, with the adoption of WSNs in real-life projects, new issues have arisen, being one of them the development of realistic strategies to deploy WSNs. We have studied different ways of solving this aspect by focusing on different optimality criteria and evaluating the different trade-offs that occur when a balanced solution must be selected. On the one hand, deterministic placements subject to conflicting goals have been addressed. Results can be obtained in the form of Pareto-frontiers, allowing proper solution selection. On the other hand, a number of situations correspond to deployments were the nodes¿ position is inherently random. We have analyzed these situations leading first to a theoretical model, which later has been particularized to a Moon WSN survey. Our work is the first considering a full model with realistic properties such as 3D topography, propellant consumptions or network lifetime and mass limitations. Furthermore, development of smart applications within IoT is the focus of the Ambient Intelligence (AmI) field. Rather than having people adapting to the surrounding environment, AmI pursues the development of sensitive environments able to anticipate support in people¿s actions. AmI is progressively being introduced in many real-life environments like education, homes, health and so forth. In this thesis we develop a sport-oriented AmI system designed to improve athletes training. The goal is developing an assistant able to provide real-time training orders based on both environment and athletes¿ biometry, which is aimed to control the aerobic and the technical-tactical training. Validation experiments with the honor league UCAM Volleyball Murcia team have shown the suitability of this approach.Universidad Politécnica de CartagenaPrograma de doctorado en Tecnología de la Información y de las Comunicacione

NASA Innovative Advanced Concepts (NIAC) Phase 1 Final Report: Venus Landsailer Zephyr

Imagine sailing across the hot plains of Venus! A design for a craft to do just this was completed by the COncurrent Multidisciplinary Preliminary Assessment of Space Systems (COMPASS) Team for the NASA Innovative Advanced Concepts (NIAC) project. The robotic craft could explore over 30 km of surface of Venus, driven by the power of the wind

Enabling Resilience in Cyber-Physical-Human Water Infrastructures

Rapid urbanization and growth in urban populations have forced community-scale infrastructures (e.g., water, power and natural gas distribution systems, and transportation networks) to operate at their limits. Aging (and failing) infrastructures around the world are becoming increasingly vulnerable to operational degradation, extreme weather, natural disasters and cyber attacks/failures. These trends have wide-ranging socioeconomic consequences and raise public safety concerns. In this thesis, we introduce the notion of cyber-physical-human infrastructures (CPHIs) - smart community-scale infrastructures that bridge technologies with physical infrastructures and people. CPHIs are highly dynamic stochastic systems characterized by complex physical models that exhibit regionwide variability and uncertainty under disruptions. Failures in these distributed settings tend to be difficult to predict and estimate, and expensive to repair. Real-time fault identification is crucial to ensure continuity of lifeline services to customers at adequate levels of quality. Emerging smart community technologies have the potential to transform our failing infrastructures into robust and resilient future CPHIs.In this thesis, we explore one such CPHI - community water infrastructures. Current urban water infrastructures, that are decades (sometimes over a 100 years) old, encompass diverse geophysical regimes. Water stress concerns include the scarcity of supply and an increase in demand due to urbanization. Deterioration and damage to the infrastructure can disrupt water service; contamination events can result in economic and public health consequences. Unfortunately, little investment has gone into modernizing this key lifeline.To enhance the resilience of water systems, we propose an integrated middleware framework for quick and accurate identification of failures in complex water networks that exhibit uncertain behavior. Our proposed approach integrates IoT-based sensing, domain-specific models and simulations with machine learning methods to identify failures (pipe breaks, contamination events). The composition of techniques results in cost-accuracy-latency tradeoffs in fault identification, inherent in CPHIs due to the constraints imposed by cyber components, physical mechanics and human operators. Three key resilience problems are addressed in this thesis; isolation of multiple faults under a small number of failures, state estimation of the water systems under extreme events such as earthquakes, and contaminant source identification in water networks using human-in-the-loop based sensing. By working with real world water agencies (WSSC, DC and LADWP, LA), we first develop an understanding of operations of water CPHI systems. We design and implement a sensor-simulation-data integration framework AquaSCALE, and apply it to localize multiple concurrent pipe failures. We use a mixture of infrastructure measurements (i.e., historical and live water pressure/flow), environmental data (i.e., weather) and human inputs (i.e., twitter feeds), combined and enhanced with the domain model and supervised learning techniques to locate multiple failures at fine levels of granularity (individual pipeline level) with detection time reduced by orders of magnitude (from hours/days to minutes). We next consider the resilience of water infrastructures under extreme events (i.e., earthquakes) - the challenge here is the lack of apriori knowledge and the increased number and severity of damages to infrastructures. We present a graphical model based approach for efficient online state estimation, where the offline graph factorization partitions a given network into disjoint subgraphs, and the belief propagation based inference is executed on-the-fly in a distributed manner on those subgraphs. Our proposed approach can isolate 80% broken pipes and 99% loss-of-service to end-users during an earthquake.Finally, we address issues of water quality - today this is a human-in-the-loop process where operators need to gather water samples for lab tests. We incorporate the necessary abstractions with event processing methods into a workflow, which iteratively selects and refines the set of potential failure points via human-driven grab sampling. Our approach utilizes Hidden Markov Model based representations for event inference, along with reinforcement learning methods for further refining event locations and reducing the cost of human efforts.The proposed techniques are integrated into a middleware architecture, which enables components to communicate/collaborate with one another. We validate our approaches through a prototype implementation with multiple real-world water networks, supply-demand patterns from water utilities and policies set by the U.S. EPA. While our focus here is on water infrastructures in a community, the developed end-to-end solution is applicable to other infrastructures and community services which operate in disruptive and resource-constrained environments

Inspection and Reconnaissance Micro-Rover for Use in Extraterrestrial Environments

The goal of this project is to design and implement a micro-rover capable of supporting a primary rover to complete mission specific tasks and objectives. This rover is designed with the intent of interfacing with many different robotic systems due to the ease of integration with Robot Operating System (ROS) and its small size. The project demonstrates the possibilities for smaller and lighter robotic rovers by exhibiting a small tele-operated, two-wheel, self-righting micro-rover with a HD video stream and sensor feedback for situational awareness designed for use in space applications. The micro-rover project proves the capabilities of creating a small inexpensive secondary rover to play a key supportive role, allowing the pair to complete mission objectives faster and more efficiently