Federated Self-Supervised Learning of Multi-Sensor Representations for Embedded Intelligence

Smartphones, wearables, and Internet of Things (IoT) devices produce a wealth of data that cannot be accumulated in a centralized repository for learning supervised models due to privacy, bandwidth limitations, and the prohibitive cost of annotations. Federated learning provides a compelling framework for learning models from decentralized data, but conventionally, it assumes the availability of labeled samples, whereas on-device data are generally either unlabeled or cannot be annotated readily through user interaction. To address these issues, we propose a self-supervised approach termed \textit{scalogram-signal correspondence learning} based on wavelet transform to learn useful representations from unlabeled sensor inputs, such as electroencephalography, blood volume pulse, accelerometer, and WiFi channel state information. Our auxiliary task requires a deep temporal neural network to determine if a given pair of a signal and its complementary viewpoint (i.e., a scalogram generated with a wavelet transform) align with each other or not through optimizing a contrastive objective. We extensively assess the quality of learned features with our multi-view strategy on diverse public datasets, achieving strong performance in all domains. We demonstrate the effectiveness of representations learned from an unlabeled input collection on downstream tasks with training a linear classifier over pretrained network, usefulness in low-data regime, transfer learning, and cross-validation. Our methodology achieves competitive performance with fully-supervised networks, and it outperforms pre-training with autoencoders in both central and federated contexts. Notably, it improves the generalization in a semi-supervised setting as it reduces the volume of labeled data required through leveraging self-supervised learning.Comment: Accepted for publication at IEEE Internet of Things Journa

Intelligent, Self-Diagnostic Thermal Protection System for Future Spacecraft

The goal of this project is to provide self-diagnostic capabilities to the thermal protection systems (TPS) of future spacecraft. Self-diagnosis is especially important in thermal protection systems (TPS), where large numbers of parts must survive extreme conditions after weeks or years in space. In-service inspections of these systems are difficult or impossible, yet their reliability must be ensured before atmospheric entry. In fact, TPS represents the greatest risk factor after propulsion for any transatmospheric mission. The concepts and much of the technology would be applicable not only to the Crew Exploration Vehicle (CEV), but also to ablative thermal protection for aerocapture and planetary exploration. Monitoring a thermal protection system on a Shuttle-sized vehicle is a daunting task: there are more than 26,000 components whose integrity must be verified with very low rates of both missed faults and false positives. The large number of monitored components precludes conventional approaches based on centralized data collection over separate wires; a distributed approach is necessary to limit the power, mass, and volume of the health monitoring system. Distributed intelligence with self-diagnosis further improves capability, scalability, robustness, and reliability of the monitoring subsystem. A distributed system of intelligent sensors can provide an assurance of the integrity of the system, diagnosis of faults, and condition-based maintenance, all with provable bounds on errors

The Propulsion of Reconfigurable Modular Robots in Fluidic Environments

Reconfigurable modular robots promise to transform the way robotic systems are designed and operated. Fluidic or microgravity environments, which can be difficult or dangerous for humans to work in, are ideal domains for the use of modular systems. This thesis proposes that combining effective propulsion, large reconfiguration space and high scalability will increase the utility of modular robots. A novel concept for the propulsion of reconfigurable modular robots is developed. Termed Modular Fluidic Propulsion (MFP), this concept describes a system that propels by routing fluid though itself. This allows MFP robots to self-propel quickly and effectively in any configuration, while featuring a cubic lattice structure. A decentralized occlusion-based motion controller for the system is developed. The simplicity of the controller, which requires neither run-time memory nor computation via logic units, combined with the simple binary sensors and actuators of the robot, gives the system a high level of scalabilty. It is proven formally that 2-D MFP robots are able to complete a directed locomotion task under certain assumptions. Simulations in 3-D show that robots composed of 125 modules in a variety of configurations can complete the task. A hardware prototype that floats on the surface of water is developed. Experiments show that robots composed of four modules can complete the task in any configuration. This thesis also investigates the evo-bots, a self-reconfigurable modular system that floats in 2-D on an air table. The evo-bot system uses a stop-start propulsion mechanism to choose between moving randomly or not moving at all. This is demonstrated experimentally for the first time. In addition, the ability of the modules to detect, harvest and share energy, as well as self-assemble into simple structures, is demonstrated

ChordMics: Acoustic Signal Purification with Distributed Microphones

Acoustic signal acts as an essential input to many systems. However, the pure acoustic signal is very difficult to extract, especially in noisy environments. Existing beamforming systems are able to extract the signal transmitted from certain directions. However, since microphones are centrally deployed, these systems have limited coverage and low spatial resolution. We overcome the above limitations and present ChordMics, a distributed beamforming system. By leveraging the spatial diversity of the distributed microphones, ChordMics is able to extract the acoustic signal from arbitrary points. To realize such a system, we further address the fundamental challenge in distributed beamforming: aligning the signals captured by distributed and unsynchronized microphones. We implement ChordMics and evaluate its performance under both LOS and NLOS scenarios. The evaluation results tell that ChordMics can deliver higher SINR than the centralized microphone array. The average performance gain is up to 15dB

Multiscouting: Guiding distributed manipulation with multiple mobile sensors

This thesis investigates the use of multiple mobile sensors to guide the motion of a distributed manipulation system. In our system, multiple robots cooperatively place a large object at a goal in a dynamic, unstructured, unmapped environment. We take the system developed in [Rus, Kabir, Kotay, Soutter 1996], which employs a single mobile sensor for navigational tasks, and extend it to allow the use of multiple mobile sensors. This allows the system to perform successful manipulations in a larger class of spaces than was possible in the single scout model. We focus on the development of a negotiation protocol that enables multiple scouts to cooperatively plan system motion. This algorithm enhances the previous\u27 system\u27s scalability and adds greater fault-tolerance. Two alternate algorithms for cooperation: a modification of negotiation and a bidding protocol, are also discussed. Finally, an implementation of the negotiation protocol is described and experimental data produced by the implementation is analyzed