Electronic systems for the restoration of the sense of touch in upper limb prosthetics

In the last few years, research on active prosthetics for upper limbs focused on improving the human functionalities and the control. New methods have been proposed for measuring the user muscle activity and translating it into the prosthesis control commands. Developing the feed-forward interface so that the prosthesis better follows the intention of the user is an important step towards improving the quality of life of people with limb amputation. However, prosthesis users can neither feel if something or someone is touching them over the prosthesis and nor perceive the temperature or roughness of objects. Prosthesis users are helped by looking at an object, but they cannot detect anything otherwise. Their sight gives them most information. Therefore, to foster the prosthesis embodiment and utility, it is necessary to have a prosthetic system that not only responds to the control signals provided by the user, but also transmits back to the user the information about the current state of the prosthesis. This thesis presents an electronic skin system to close the loop in prostheses towards the restoration of the sense of touch in prosthesis users. The proposed electronic skin system inlcudes an advanced distributed sensing (electronic skin), a system for (i) signal conditioning, (ii) data acquisition, and (iii) data processing, and a stimulation system. The idea is to integrate all these components into a myoelectric prosthesis. Embedding the electronic system and the sensing materials is a critical issue on the way of development of new prostheses. In particular, processing the data, originated from the electronic skin, into low- or high-level information is the key issue to be addressed by the embedded electronic system. Recently, it has been proved that the Machine Learning is a promising approach in processing tactile sensors information. Many studies have been shown the Machine Learning eectiveness in the classication of input touch modalities.More specically, this thesis is focused on the stimulation system, allowing the communication of a mechanical interaction from the electronic skin to prosthesis users, and the dedicated implementation of algorithms for processing tactile data originating from the electronic skin. On system level, the thesis provides design of the experimental setup, experimental protocol, and of algorithms to process tactile data. On architectural level, the thesis proposes a design ow for the implementation of digital circuits for both FPGA and integrated circuits, and techniques for the power management of embedded systems for Machine Learning algorithms

Human and Biological Skin-Inspired Electronic Skins for Advanced Sensory Functions and Multifunctionality

Department of Energy Engineering (Energy Engineering)The electronic skin (e-skin) technology is an exciting frontier to drive next generation of wearable electronics owing to its high level of wearability to curved human body, enabling high accuracy to harvest information of users and their surroundings. Altough various types of e-skins, based on several signal-transduction modes, including piezoresistive, capacitive, piezoelectric, triboelectric modes, have been developed, their performances (i.e. sensitivity, working range, linearity, multifunctionality, etc.) should be improved for the wearable applications. Recently, biomimicry of the human and biological skins has become a great inspiration for realizing novel wearable e-skin systems with exceptional multifunctionality as well as advanced sensory functions. As an ideal sensory organ, tactile sensing capabilities of human skin was emulated for the development of e-skins with enhanced sensor performances. In particular, the unique geometry and systematic sensory system of human skin have driven new opportunities in multifunctional and highly sensitive e-skin applications. In addition, extraordinary architectures for protection, locomotion, risk indication, and camouflage in biological systems provide great possibilities for second skin applications on user-interactive, skin-attachable, and ultrasensitive e-skins, as well as soft robots. Benefitting from their superior perceptive functions and multifunctionality, human and biological skins-inspired e-skins can be considered to be promising candidates for wearable device applications, such as body motion tracking, healthcare devices, acoustic sensor, and human machine interfaces (HMI). This thesis covers our recent studies about human and biological skin-inspired e-skins for advanced sensory functions and multifunctionality. First, chapter 1 highlights various types of e-skins and recent research trends in bioinspired e-skins mimicking perceptive features of human and biological skins. In chapter 2, we demonstrate highly sensitive and tactile-direction-sensitive e-skin based on human skin-inspired interlocked microdome structures. Owing to the stress concentration effect, the interlocked e-skin experiences significant change of contact area between the interlocked microdomes, resulting in high pressure sensitivity. In addition, because of the different deformation trends between microstructures in mutual contact, the interlocked e-skin can differentiate and decouple sensor signals under different directional forces, such as pressure, tensile strain, shear, and bending. In chapter 3, interlocked e-skins were designed with multilayered geometry. Although interlocked e-skin shows highly sensitive pressure sensing performances, their pressure sensing range is narrow and pressure sensitivity continuously decreases with increasing pressure level. The multilayer interlocked microdome geometry can enhance the pressure-sensing performances of e-skins, such as sensitivity, working range, and linearity. As another approach of e-skin with multilayered geometry, we demonstrate multilayered e-skin based on conductivity-gradient conductive materials in chapter 4. The conducive polymer composites with different conductivity were coated on the microdome pattern and designed as interlocked e-skin with coplanar electrode design, resulting in exceptionally high pressure-sensing performances compared with previous literatures. In chapter 5, inspired by responsive color change in biological skins, we developed mechanochromic e-skin with a hierarchical nanoparticle-in-micropore architecture. The novel design of hierarchical structure enables effective stress concentration at the interface between nanoparticle and porous structure, resulting in impressive color change under mechanical stimuli. In chapter 6, we emulate ultrahigh temperature sensitivity of human and snake skin for temperature-sensitive e-skin. The thermoresponsive composite based on semi-crystalline polymer, temperature sensor shows ultrahigh temperature sensitivity near the melting point of semi-crystalline polymer. In addition, integration of thermochromic composite, mimicking biological skins, enables dual-mode temperature sensors by electrical and colorimetric sensing capabilities. Finally, in chapter 7, we summarize this thesis along with future perspective that should be considered for next-generation e-skin electronics. Our e-skins, inspired by human and biological skin, can provide a new paradigm for realizing novel wearable electronic systems with exceptional multifunctionality as well as advanced sensory functions.clos

MECHANOSENSORY FEEDBACK FOR FLIGHT CONTROL AND PREY CAPTURE IN THE ECHOLOCATING BAT

Throughout the animal kingdom, organisms have evolved neural systems that process biologically relevant stimuli to guide a wide range of species-specific behaviors. Bats, comprising 25% of mammalian species, rely on diverse sensory modalities to carry out tasks such as foraging, obstacle avoidance and social communication. While it is well known that many bat species use echolocation to find food and steer around obstacles, they also depend on other senses. For instance, some bats predominantly use vision to navigate, and others use olfaction to find food sources. In addition, bats rely on airflow sensors to stabilize their flight, primarily through signals carried by microscopic hairs embedded in their wings and tail membranes. Studies have shown that bats performing an obstacle avoidance task show changes in their flight behavior when dorsal wing hairs are removed. Additionally, electrophysiological studies have shown that wing hairs are involved in airflow sensing, but little is known about the contribution of sensory hairs on the ventral surfaces of the wing and tail membranes to their flight control and other complex behaviors, such as prey handling. Chapter 1 of my dissertation presents a general introduction to bat echolocation, flight kinematics, and airflow sensing for flight control. In Chapter 2, I review sensory hairs across the animal kingdom, from invertebrates to vertebrates. I discuss the role of sensory hairs for functions ranging from detection to locomotion and propose the use and benefit of mechanosensors in biologically-inspired technology. In Chapter 3, I devised an experiment to evaluate changes in capture success, as well changes in flight kinematics and adaptive sonar behavior, before and after depilation of sensory hairs in order to ascertain if these sensory hairs have a functional role in both airflow sensing for flight control and tactile sensing for prey handling. In Chapter 4, I designed an experiment aimed at determining if firing patterns of S1 neurons change with airflow speed and angle of attack and if wing hair depilation affects S1 responses to whole wing stimulation. To answer these questions, I record neural activity in S1 of sedated big brown bats while the entire contralateral wing is systematically exposed to naturalistic airflow in a wind tunnel. Finally, in Chapter 5, I address open questions that remain, present experiments aimed at filling these gaps, and consider key points important for future work

Somatic ABC's: A Theoretical Framework for Designing, Developing and Evaluating the Building Blocks of Touch-Based Information Delivery

abstract: Situations of sensory overload are steadily becoming more frequent as the ubiquity of technology approaches reality--particularly with the advent of socio-communicative smartphone applications, and pervasive, high speed wireless networks. Although the ease of accessing information has improved our communication effectiveness and efficiency, our visual and auditory modalities--those modalities that today's computerized devices and displays largely engage--have become overloaded, creating possibilities for distractions, delays and high cognitive load; which in turn can lead to a loss of situational awareness, increasing chances for life threatening situations such as texting while driving. Surprisingly, alternative modalities for information delivery have seen little exploration. Touch, in particular, is a promising candidate given that it is our largest sensory organ with impressive spatial and temporal acuity. Although some approaches have been proposed for touch-based information delivery, they are not without limitations including high learning curves, limited applicability and/or limited expression. This is largely due to the lack of a versatile, comprehensive design theory--specifically, a theory that addresses the design of touch-based building blocks for expandable, efficient, rich and robust touch languages that are easy to learn and use. Moreover, beyond design, there is a lack of implementation and evaluation theories for such languages. To overcome these limitations, a unified, theoretical framework, inspired by natural, spoken language, is proposed called Somatic ABC's for Articulating (designing), Building (developing) and Confirming (evaluating) touch-based languages. To evaluate the usefulness of Somatic ABC's, its design, implementation and evaluation theories were applied to create communication languages for two very unique application areas: audio described movies and motor learning. These applications were chosen as they presented opportunities for complementing communication by offloading information, typically conveyed visually and/or aurally, to the skin. For both studies, it was found that Somatic ABC's aided the design, development and evaluation of rich somatic languages with distinct and natural communication units.Dissertation/ThesisPh.D. Computer Science 201

Design, characterization and validation of integrated bioelectronics for cellular studies: from inkjet-printed sensors to organic actuators

Mención Internacional en el título de doctorAdvances in bioinspired and biomimetic electronics have enabled coupling engineering devices to biological systems with unprecedented integration levels. Major efforts, however, have been devoted to interface malleable electronic devices externally to the organs and tissues. A promising alternative is embedding electronics into living tissues/organs or, turning the concept inside out, lading electronic devices with soft living matters which may accomplish remote monitoring and control of tissue’s functions from within. This endeavor may unleash the ability to engineer “living electronics” for regenerative medicine and biomedical applications. In this context, it remains a challenge to insert electronic devices efficiently with living cells in a way that there are minimal adverse reactions in the biological host while the electronics maintaining the engineered functionalities. In addition, investigating in real-time and with minimal invasion the long-term responses of biological systems that are brought in contact with such bioelectronic devices is desirable. In this work we introduce the development (design, fabrication and characterization) and validation of sensors and actuators mechanically soft and compliant to cells able to properly operate embedded into a cell culture environment, specifically of a cell line of human epithelial keratinocytes. For the development of the sensors we propose moving from conventional microtechnology approaches to techniques compatible with bioprinting in a way to support the eventual fabrication of tissues and electronic sensors in a single hybrid plataform simultaneously. For the actuators we explore the use of electroactive, organic, printing-compatible polymers to induce cellular responses as a drug-free alternative to the classic chemical route in a way to gain eventual control of biological behaviors electronically. In particular, the presented work introduces inkjet-printed interdigitated electrodes to monitor label-freely and non-invasively cellular migration, proliferation and cell-sensor adhesions of epidermal cells (HaCaT cells) using impedance spectroscopy and the effects of (dynamic) mechanical stimulation on proliferation, migration and morphology of keratinocytes by varying the magnitude, frequency and duration of mechanical stimuli exploiting the developed biocompatible actuator. The results of this thesis contribute to the envision of three-dimensional laboratory-growth tissues with built-in electronics, paving exciting avenues towards the idea of living smart cyborg-skin substitutes.En los útimos años los avances en el desarrollo de dispositivos electrónicos diseñados imitando las propiedades de sistemas vivos han logrado acoplar sistemas electrónicos y órganos/tejidos biológicos con un nivel de integración sin precedentes. Convencionalmente, la forma en que estos sistemas bioelectrónicos son integrados con órganos o tejidos ha sido a través del contacto superficial entre ambos sistemas, es decir acoplando la electrónica externamente al tejido. Lamentablemente estas aproximaciones no contemplan escenarios donde ha habido una pérdida o daño del tejido con el cual interactuar, como es el caso de daños en la piel debido a quemaduras, úlceras u otras lesiones genéticas o producidas. Una alternativa prometedora para ingeniería de tejidos y medicina regenerativa, y en particular para implantes de piel, es embeber la electrónica dentro del tejido, o presentado de otra manera, cargar el sistema electrónico con células vivas y tejidos fabricados por ingeniería de tejidos como parte innata del propio dispositivo. Este concepto permitiría no solo una monitorización remota y un control basado en señalizaciones eléctricas (sin químicos) de tejidos biológicos fabricados mediante técnicas de bioingeniería desde dentro del propio tejido, sino también la fabricación de una “electrónica viva”, biológica y eléctricamente funcional. En este contexto, es un desafío insertar de manera eficiente dispositivos electrónicos con células vivas sin desencadenar reacciones adversas en el sistema biológico receptor ni en el sistema electrónico diseñado. Además, es deseable monitorizar en tiempo real y de manera mínimamente invasiva las respuestas de dichos sistemas biológicos que se han añadido a tales dispositivos bioelectrónicos. En este trabajo presentamos el desarrollo (diseño, fabricación y caracterización) y validación de sensores y actuadores mecánicamente suaves y compatibles con células capaces de funcionar correctamente dentro de un entorno de cultivo celular, específicamente de una línea celular de células epiteliales humanas. Para el desarrollo de los sensores hemos propuesto utilizar técnicas compatibles con la bioimpresión, alejándonos de la micro fabricación tradicionalmente usada para la manufactura de sensores electrónicos, con el objetivo a largo plazo de promover la fabricación de los tejidos y los sensores electrónicos simultáneamente en un mismo sistema de impresión híbrido. Para el desarrollo de los actuadores hemos explorado el uso de polímeros electroactivos y compatibles con impresión y hemos investigado el efecto de estímulos mecánicos dinámicos en respuestas celulares con el objetivo a largo plazo de autoinducir comportamientos biológicos controlados de forma electrónica. En concreto, este trabajo presenta sensores basados en electrodos interdigitados impresos por inyección de tinta para monitorear la migración celular, proliferación y adhesiones célula-sustrato de una línea celular de células epiteliales humanas (HaCaT) en tiempo real y de manera no invasiva mediante espectroscopía de impedancia. Por otro lado, este trabajo presenta actuadores biocompatibles basados en el polímero piezoeléctrico fluoruro de poli vinilideno y ha investigado los efectos de estimular mecánicamente células epiteliales en relación con la proliferación, migración y morfología celular mediante variaciones dinámicas de la magnitud, frecuencia y duración de estímulos mecánicos explotando el actuador biocompatible propuesto. Ambos sistemas presentados como resultado de esta tesis doctoral contribuyen al desarrollo de tejidos 3D con electrónica incorporada, promoviendo una investigación hacia la fabricación de sustitutos equivalentes de piel mitad orgánica mitad electrónica como tejidos funcionales biónicos inteligentes.The main works presented in this thesis have been conducted in the facilities of the Universidad Carlos III de Madrid with support from the program Formación del Profesorado Universitario FPU015/06208 granted by Spanish Ministry of Education, Culture and Sports. Some of the work has been also developed in the facilities of the Fraunhofer-Institut für Zuverlässigkeit und Mikrointegration (IZM) and University of Applied Sciences (HTW) in Berlin, under the supervision of Prof. Dr. Ing. H-D. Ngo during a research visit funded by the Mobility Fellows Program by the Spanish Ministry of Education, Culture, and Sports. This work has been developed in the framework of the projects BIOPIELTEC-CM (P2018/BAA-4480), funded by Comunidad de Madrid, and PARAQUA (TEC2017-86271-R) funded by Ministerio de Ciencia e Innovación.Programa de Doctorado en Ingeniería Eléctrica, Electrónica y Automática por la Universidad Carlos III de MadridPresidente: José Antonio García Souto.- Secretario: Carlos Elvira Pujalte.- Vocal: María Dimak