A sub-mW IoT-endnode for always-on visual monitoring and smart triggering

This work presents a fully-programmable Internet of Things (IoT) visual sensing node that targets sub-mW power consumption in always-on monitoring scenarios. The system features a spatial-contrast $128\mathrm{x}64$ binary pixel imager with focal-plane processing. The sensor, when working at its lowest power mode ($10\mu W$ at 10 fps), provides as output the number of changed pixels. Based on this information, a dedicated camera interface, implemented on a low-power FPGA, wakes up an ultra-low-power parallel processing unit to extract context-aware visual information. We evaluate the smart sensor on three always-on visual triggering application scenarios. Triggering accuracy comparable to RGB image sensors is achieved at nominal lighting conditions, while consuming an average power between $193\mu W$ and $277\mu W$, depending on context activity. The digital sub-system is extremely flexible, thanks to a fully-programmable digital signal processing engine, but still achieves 19x lower power consumption compared to MCU-based cameras with significantly lower on-board computing capabilities.Comment: 11 pages, 9 figures, submitteted to IEEE IoT Journa

Design of a Low-Power Automatic Wireless Multi-Logger Networking Device

Virtually every industry and discipline (e.g., mining, pharmaceutical, construction, agriculture, reclamation, etc.) is finding applications for wireless data acquisition for monitoring and managing processes and resources. Two sectors, namely agriculture and environmental research, are seeking ways to obtain distributed soil and plant measurements over larger areas like a watershed or large fields rather than a single site of intensive instrumentation (i.e., a weather station). Wireless sensor networks and remote sensing have been explored as a means to satisfy this need. Commercial products are readily available that have remote wireless options to support distributed senor networking. However, these systems have been designed with a field engineer or technician as the target end-user. Equipment and operating costs, device specific programming languages, and complex wireless configuration schemes have impeded the adoption of large-scale, multi-node wireless systems in these fields. This report details the development of an easy-to-use, ultra-low power wireless datalogger incorporating a scalable, intelligent data collection and transmission topology. The final product can interface to various sensor types including SDI-12 and uses an LCD display to help simplify device setup

Standard interface definition for avionics data bus systems

Data bus for avionics system of space shuttle, noting functions of interface unit, error detection and recovery, redundancy, and bus control philosoph

Programming frameworks for mobile sensing

The proliferation of smart mobile devices in people’s daily lives is making context-aware computing a reality. A plethora of sensors available in these devices can be utilized to understand users’ context better. Apps can provide more relevant data or services to the user based on improved understanding of user’s context. With the advent of cloud-assisted mobile platforms, apps can also perform collaborative computation over the sensing data collected from a group of users. However, there are still two main issues: (1) A lack of simple and effective personal sensing frameworks: existing frameworks do not provide support for real-time fusing of data from motion and visual sensors in a simple manner, and no existing framework collectively utilizes sensors from multiple personal devices and personal IoT sensors, and (2) a lack of collaborative/distributed computing frameworks for mobile users. This dissertation presents solutions for these two issues. The first issue is addressed by TagPix and Sentio, two frameworks for mobile sensing. The second issue is addressed by Moitree, a middleware for mobile distributed computing, and CASINO, a collaborative sensor-driven offloading system. TagPix is a real-time, privacy preserving photo tagging framework, which works locally on the phones and consumes little resources (e.g., battery). It generates relevant tags for landscape photos by utilizing sensors of a mobile device and it does not require any previous training or indexing. When a user aims the mobile camera to a particular landmark, the framework uses accelerometer and geomagnetic field sensor to identify in which direction the user is aiming the camera at. It then uses a landmark database and employs a smart distance estimation algorithm to identify which landmark(s) is targeted by the user. The framework then generates relevant tags for the captured photo using these information. A more versatile sensing framework can be developed using sensors from multiple devices possessed by a user. Sentio is such a framework which enables apps to seamlessly utilize the collective sensing capabilities of the user’s personal devices and of the IoT sensors located in the proximity of the user. With Sentio, an app running on any personal mobile/wearable device can access any sensor of the user in real-time using the same API, can selectively switch to the most suitable sensor of a particular type when multiple sensors of this type are available at different devices, and can build composite sensors. Sentio offers seamless connectivity to sensors even if the sensor-accessing code is offloaded to the cloud. Sentio provides these functionalities with a high-level API and a distributed middleware that handles all low-level communication and sensor management tasks. This dissertation also proposes Moitree, a middleware for the mobile cloud platforms where each mobile device is augmented by an avatar, a per-user always-on software entity that resides in the cloud. Mobile-avatar pairs participate in distributed computing as a unified computing entity. Moitree provides a common programming and execution framework for mobile distributed apps. Moitree allows the components of a distributed app to execute seamlessly over a set of mobile/avatar pairs, with the provision of offloading computation and communication to the cloud. The programming framework has two key features: user collaborations are modeled using group semantics - groups are created dynamically based on context and are hierarchical; data communication among group members is offloaded to the cloud through high-level communication channels. Finally, this dissertation presents and discusses CASINO, a collaborative sensor-driven computation offloading framework which can be used alongside Moitree. This framework includes a new scheduling algorithm which minimizes the total completion time of a collaborative computation that executes over a set of mobile/avatar pairs. Using the CASINO API, the programmers can mark their classes and functions as ”offloadable”. The framework collects profiling information (network, CPU, battery, etc.) from participating users’ mobile devices and avatars, and then schedules ”offloadable” tasks in mobiles and avatars in a way that reduces the total completion time. The scheduling problem is proven to be NP-Hard and there is no polynomial time optimization algorithm for it. The proposed algorithm can generate a schedule in polynomial time using a topological sorting and greedy technique

Leveraging Intelligent Computation Offloading with Fog/Edge Computing for Tactile Internet: Advantages and Limitations

[EN] With the recent advancement in wireless communication and networks, we are at the doorstep of the Tactile Internet. The Tactile Internet aims to enable the skill delivery and thereafter democratize the specialized skills for many emerging applications (e.g., remote medical, industrial machinery, remote robotics, autonomous driving). In this article, we start with the motivation of applying intelligent edge computing for computation offloading in the Tactile Internet. Afterward, we outline the main research challenges to leverage edge intelligence at the master, network, and controlled domain of the Tactile Internet. The key research challenges in the Tactile Internet lie in its stringent requirements such as ultra-low latency, ultra-high reliability, and almost zero service outage. We also discuss major entities in intelligent edge computing and their role in the Tactile Internet. Finally, several potential research challenges in edge intelligence for the Tactile Internet are highlighted.This work was supported in part by the National Natural Science Foundation of China under Grant 61901128, and Agile Edge Intelligence for Delay Sensitive IoT (AgilE-IoT) project (Grant No. 9131-00119B) of Independent Research Fund Denmark (DFF).Mukherjee, M.; Guo, M.; Lloret, J.; Zhang, Q. (2020). Leveraging Intelligent Computation Offloading with Fog/Edge Computing for Tactile Internet: Advantages and Limitations. IEEE Network. 34(5):322-329. https://doi.org/10.1109/MNET.001.200000432232934

To Compute or not to Compute? Adaptive Smart Sensing in Resource-Constrained Edge Computing

We consider a network of smart sensors for edge computing application that sample a signal of interest and send updates to a base station for remote global monitoring. Sensors are equipped with sensing and compute, and can either send raw data or process them on-board before transmission. Limited hardware resources at the edge generate a fundamental latency-accuracy trade-off: raw measurements are inaccurate but timely, whereas accurate processed updates are available after computational delay. Also, if sensor on-board processing entails data compression, latency caused by wireless communication might be higher for raw measurements. Hence, one needs to decide when sensors should transmit raw measurements or rely on local processing to maximize overall network performance. To tackle this sensing design problem, we model an estimation-theoretic optimization framework that embeds computation and communication delays, and propose a Reinforcement Learning-based approach to dynamically allocate computational resources at each sensor. Effectiveness of our proposed approach is validated through numerical simulations with case studies motivated by the Internet of Drones and self-driving vehicles.Comment: 14 pages, 14 figures; submitted to IEEE TNSM; revised versio

A geographically distributed bio-hybrid neural network with memristive plasticity

Throughout evolution the brain has mastered the art of processing real-world inputs through networks of interlinked spiking neurons. Synapses have emerged as key elements that, owing to their plasticity, are merging neuron-to-neuron signalling with memory storage and computation. Electronics has made important steps in emulating neurons through neuromorphic circuits and synapses with nanoscale memristors, yet novel applications that interlink them in heterogeneous bio-inspired and bio-hybrid architectures are just beginning to materialise. The use of memristive technologies in brain-inspired architectures for computing or for sensing spiking activity of biological neurons8 are only recent examples, however interlinking brain and electronic neurons through plasticity-driven synaptic elements has remained so far in the realm of the imagination. Here, we demonstrate a bio-hybrid neural network (bNN) where memristors work as "synaptors" between rat neural circuits and VLSI neurons. The two fundamental synaptors, from artificial-to-biological (ABsyn) and from biological-to- artificial (BAsyn), are interconnected over the Internet. The bNN extends across Europe, collapsing spatial boundaries existing in natural brain networks and laying the foundations of a new geographically distributed and evolving architecture: the Internet of Neuro-electronics (IoN).Comment: 16 pages, 10 figure