Finding Common Ground: A Survey of Capacitive Sensing in Human-Computer Interaction

For more than two decades, capacitive sensing has played a prominent role in human-computer interaction research. Capacitive sensing has become ubiquitous on mobile, wearable, and stationary devices---enabling fundamentally new interaction techniques on, above, and around them. The research community has also enabled human position estimation and whole-body gestural interaction in instrumented environments. However, the broad field of capacitive sensing research has become fragmented by different approaches and terminology used across the various domains. This paper strives to unify the field by advocating consistent terminology and proposing a new taxonomy to classify capacitive sensing approaches. Our extensive survey provides an analysis and review of past research and identifies challenges for future work. We aim to create a common understanding within the field of human-computer interaction, for researchers and practitioners alike, and to stimulate and facilitate future research in capacitive sensing

Real-time Neuromorphic Visual Pre-Processing and Dynamic Saliency

The human brain is by far the most computationally complex, efficient, and reliable computing system operating under such low-power, small-size, and light-weight specifications. Within the field of neuromorphic engineering, we seek to design systems with facsimiles to that of the human brain with means to reach its desirable properties. In this doctoral work, the focus is within the realm of vision, specifically visual saliency and related visual tasks with bio-inspired, real-time processing. The human visual system, from the retina through the visual cortical hierarchy, is responsible for extracting visual information and processing this information, forming our visual perception. This visual information is transmitted through these various layers of the visual system via spikes (or action potentials), representing information in the temporal domain. The objective is to exploit this neurological communication protocol and functionality within the systems we design. This approach is essential for the advancement of autonomous, mobile agents (i.e. drones/MAVs, cars) which must perform visual tasks under size and power constraints in which traditional CPU or GPU implementations to not suffice. Although the high-level objective is to design a complete visual processor with direct physical and functional correlates to the human visual system, we focus on three specific tasks. The first focus of this thesis is the integration of motion into a biologically-plausible proto-object-based visual saliency model. Laurent Itti, one of the pioneers in the field, defines visual saliency as ``the distinct subjective perceptual quality which makes some items in the world stand out from their neighbors and immediately grab our attention.'' From humans to insects, visual saliency is important for the extraction of only interesting regions of visual stimuli for further processing. Prior to this doctoral work, Russel et al. \cite{russell2014model} designed a model of proto-object-based visual saliency with biological correlates. This model was designed for computing saliency only on static images. However, motion is a naturally occurring phenomena that plays an essential role in both human and animal visual processing. Henceforth, the most ideal model of visual saliency should consider motion that may be exhibited within the visual scene. In this work a novel dynamic proto-object-based visual saliency is described which extends the Russel et. al. saliency model to consider not only static, but also temporal information. This model was validated by using metrics for determining how accurate the model is in predicting human eye fixations and saccades on a public dataset of videos with attached eye tracking data. This model outperformed other state-of-the-art visual saliency models in computing dynamic visual saliency. Such a model that can accurately predict where humans look, can serve as a front-end component to other visual processors performing tasks such as object detection and recognition, or object tracking. In doing so it can reduce throughput and increase processing speed for such tasks. Furthermore, it has more obvious applications in artificial intelligence in mimicking the functionality of the human visual system. The second focus of this thesis is the implementation of this visual saliency model on an FPGA (Field Programmable Gate Array) for real-time processing. Initially, this model was designed within MATLAB, a software-based approach running on a CPU, which limits the processing speed and consumes unnecessary amounts of power due to overhead. This is detrimental for integration with an autonomous, mobile system which must operate in real-time. This novel FPGA implementation allows for a low-power, high-speed approach to computing visual saliency. There are a few existing FPGA-based implementations of visual saliency, and of those, none are based on the notion of proto-objects. This work presents the first, to our knowledge, FPGA implementation of an object-based visual saliency model. Such an FPGA implementation allows for the low-power, light-weight, and small-size specifications that we seek within the field of neuromorphic engineering. For validating the FPGA model, the same metrics are used for determining the extent to which it predicts human eye saccades and fixations. We compare this hardware implementation to the software model for validation. The third focus of this thesis is the design of a generic neuromorphic platform both on FPGA and VLSI (Very-Large-Scale-Integration) technology for performing visual tasks, including those necessary in the computation of the visual saliency. Visual processing tasks such as image filtering and image dewarping are demonstrated via this novel neuromorphic technology consisting of an array of hardware-based generalized integrate-and-fire neurons. It allows the visual saliency model's computation to be offloaded onto this hardware-based architecture. We first demonstrate an emulation of this neuromorphic system on FPGA demonstrating its capability of dewarping and filtering tasks as well as integration with a neuromorphic camera called the ATIS (Asynchronous Time-based Image Sensor). We then demonstrate the neuromorphic platform implemented in CMOS technology, specifically designed for low-mismatch, high-density, and low-power. Such a VLSI technology-based platform further bridges the gap between engineering and biology and moves us closer towards developing a complete neuromorphic visual processor

Simulation verification techniques study. Subsystem simulation validation techniques

Techniques for validation of software modules which simulate spacecraft onboard systems are discussed. An overview of the simulation software hierarchy for a shuttle mission simulator is provided. A set of guidelines for the identification of subsystem/module performance parameters and critical performance parameters are presented. Various sources of reference data to serve as standards of performance for simulation validation are identified. Environment, crew station, vehicle configuration, and vehicle dynamics simulation software are briefly discussed from the point of view of their interfaces with subsystem simulation modules. A detailed presentation of results in the area of vehicle subsystems simulation modules is included. A list of references, conclusions and recommendations are also given

VLSI Circuits for Bidirectional Neural Interfaces

Medical devices that deliver electrical stimulation to neural tissue are important clinical tools that can augment or replace pharmacological therapies. The success of such devices has led to an explosion of interest in the field, termed neuromodulation, with a diverse set of disorders being targeted for device-based treatment. Nevertheless, a large degree of uncertainty surrounds how and why these devices are effective. This uncertainty limits the ability to optimize therapy and gives rise to deleterious side effects. An emerging approach to improve neuromodulation efficacy and to better understand its mechanisms is to record bioelectric activity during stimulation. Understanding how stimulation affects electrophysiology can provide insights into disease, and also provides a feedback signal to autonomously tune stimulation parameters to improve efficacy or decrease side-effects. The aims of this work were taken up to advance the state-of-the-art in neuro-interface technology to enable closed-loop neuromodulation therapies. Long term monitoring of neuronal activity in awake and behaving subjects can provide critical insights into brain dynamics that can inform system-level design of closed-loop neuromodulation systems. Thus, first we designed a system that wirelessly telemetered electrocorticography signals from awake-behaving rats. We hypothesized that such a system could be useful for detecting sporadic but clinically relevant electrophysiological events. In an 18-hour, overnight recording, seizure activity was detected in a pre-clinical rodent model of global ischemic brain injury. We subsequently turned to the design of neurostimulation circuits. Three critical features of neurostimulation devices are safety, programmability, and specificity. We conceived and implemented a neurostimulator architecture that utilizes a compact on-chip circuit for charge balancing (safety), digital-to-analog converter calibration (programmability) and current steering (specificity). Charge balancing accuracy was measured at better than 0.3%, the digital-to-analog converters achieved 8-bit resolution, and physiological effects of current steering stimulation were demonstrated in an anesthetized rat. Lastly, to implement a bidirectional neural interface, both the recording and stimulation circuits were fabricated on a single chip. In doing so, we implemented a low noise, ultra-low power recording front end with a high dynamic range. The recording circuits achieved a signal-to-noise ratio of 58 dB and a spurious-free dynamic range of better than 70 dB, while consuming 5.5 μW per channel. We demonstrated bidirectional operation of the chip by recording cardiac modulation induced through vagus nerve stimulation, and demonstrated closed-loop control of cardiac rhythm

Nonlinear vibration energy harvesters for powering the internet of things

The ever decreasing power consumption in electronic devices and sensors have facilitated the development of autonomous wireless sensor nodes (WSNs), which ushered in the era of the Internet of Things (IoT). However, the problem of long-term power supply to the numerous WSNs pervasively dispersed to enable the IoT is yet to be resolved. This work focuses on the development of novel vibration energy harvesting (VEH) devices and technologies for effective transduction of mostly wide-band and noisy ambient mechanical vibrations to power WSNs. In this thesis meso-scale and MEMS-scale nonlinear and frequency tunable VEH devices have been designed, fabricated and characterized. The first meso-scale VEH prototype developed in this thesis combines a nonlinear bistable oscillator with mechanical impact induced nonlinearity, which exhibits upto 118% broadening in the frequency response over a standalone bistable system. The second meso-scale prototype combines magnetic repulsion induced bistable nonlinearity with stretching induced monostable cubic nonlinearity in a single device structure. The device effectively merged the beneficial features of the individual nonlinear bistable and monostable systems, and demonstrates upto 85% enhanced spectral performance compared to the bistable device. The third prototype is a MEMS-scale device fabricated using spiral silicon spring structure and double-layer planar micro-coils. A magnetic repulsion induced frequency tuning mechanism was incorporated in the prototype, and it was demonstrated that both linear and nonlinear hysteretic frequency responses could be tuned (by upto 18.6%) to match various ambient vibration frequencies. In order to enhance the power generating capability of MEMS-scale electromagnetic devices, an ultra-dense multi-layer micro-coil architecture has been developed. The proposed ultra-dense micro-coil is designed to incorporate double number of turns within the same volume as a conventional micro-coil, and significantly enhance the magnetic flux linkage gradient resulting in higher power output (~4 times). However, attempts to fabricate the ultra-dense coil have not been successful due to lack of proper insulation between the successive coil layers. Finally, a power management system combining diode equivalent low voltage drop (DELVD) circuit and a boost regulator module was developed. It was demonstrated that energy harvested from harmonic and bandlimited random vibrations using linear, nonlinear bistable, and combined nonlinear VEH devices could be conditioned into usable electricity by the power management system with 60% - 75% efficiency. In addition to developing new prototypes and techniques, this thesis recommends directions towards future research for further improvement in vibration energy harvesting devices and technologies