Tactile and Touchless Sensors Printed on Flexible Textile Substrates for Gesture Recognition

Tesis por compendio[EN] The main objective of this thesis is the development of new sensors and actuators using Printed Electronics technology. For this, conductive, semiconductor and dielectric polymeric materials are used on flexible and/or elastic substrates. By means of suitable designs and application processes, it is possible to manufacture sensors capable of interacting with the environment. In this way, specific sensing functionalities can be incorporated into the substrates, such as textile fabrics. Additionally, it is necessary to include electronic systems capable of processing the data obtained, as well as its registration. In the development of these sensors and actuators, the physical properties of the different materials are precisely combined. For this, multilayer structures are designed where the properties of some materials interact with those of others. The result is a sensor capable of capturing physical variations of the environment, and convert them into signals that can be processed, and finally transformed into data. On the one hand, a tactile sensor printed on textile substrate for 2D gesture recognition was developed. This sensor consists of a matrix composed of small capacitive sensors based on a capacitor type structure. These sensors were designed in such a way that, if a finger or other object with capacitive properties, gets close enough, its behaviour varies, and it can be measured. The small sensors are arranged in this matrix as in a grid. Each sensor has a position that is determined by a row and a column. The capacity of each small sensor is periodically measured in order to assess whether significant variations have been produced. For this, it is necessary to convert the sensor capacity into a value that is subsequently digitally processed. On the other hand, to improve the effectiveness in the use of the developed 2D touch sensors, the way of incorporating an actuator system was studied. Thereby, the user receives feedback that the order or action was recognized. To achieve this, the capacitive sensor grid was complemented with an electroluminescent screen printed as well. The final prototype offers a solution that combines a 2D tactile sensor with an electroluminescent actuator on a printed textile substrate. Next, the development of a 3D gesture sensor was carried out using a combination of sensors also printed on textile substrate. In this type of 3D sensor, a signal is sent generating an electric field on the sensors. This is done using a transmission electrode located very close to them. The generated field is received by the reception sensors and converted to electrical signals. For this, the sensors are based on electrodes that act as receivers. If a person places their hands within the emission area, a disturbance of the electric field lines is created. This is due to the deviation of the lines to ground using the intrinsic conductivity of the human body. This disturbance affects the signals received by the electrodes. Variations captured by all electrodes are processed together and can determine the position and movement of the hand on the sensor surface. Finally, the development of an improved 3D gesture sensor was carried out. As in the previous development, the sensor allows contactless gesture detection, but increasing the detection range. In addition to printed electronic technology, two other textile manufacturing technologies were evaluated.[ES] La presente tesis doctoral tiene como objetivo fundamental el desarrollo de nuevos sensores y actuadores empleando la tecnología electrónica impresa, también conocida como Printed Electronics. Para ello, se emplean materiales poliméricos conductores, semiconductores y dieléctricos sobre sustratos flexibles y/o elásticos. Por medio de diseños y procesos de aplicación adecuados, es posible fabricar sensores capaces de interactuar con el entorno. De este modo, se pueden incorporar a los sustratos, como puedan ser tejidos textiles, funcionalidades específicas de medición del entorno y de respuesta ante cambios de este. Adicionalmente, es necesario incluir sistemas electrónicos, capaces de realizar el procesado de los datos obtenidos, así como de su registro. En el desarrollo de estos sensores y actuadores se combinan las propiedades físicas de los diferentes materiales de forma precisa. Para ello, se diseñan estructuras multicapa donde las propiedades de unos materiales interaccionan con las de los demás. El resultado es un sensor capaz de captar variaciones físicas del entorno, y convertirlas en señales que pueden ser procesadas y transformadas finalmente en datos. Por una parte, se ha desarrollado un sensor táctil impreso sobre sustrato textil para reconocimiento de gestos en 2D. Este sensor se compone de una matriz formada por pequeños sensores capacitivos basados en estructura de tipo condensador. Estos se han diseñado de forma que, si un dedo u otro objeto con propiedades capacitivas se aproxima suficientemente, su comportamiento varía, pudiendo ser medido. Los pequeños sensores están ordenados en dicha matriz como en una cuadrícula. Cada sensor tiene una posición que viene determinada por una fila y por una columna. Periódicamente se mide la capacidad de cada pequeño sensor con el fin de evaluar si ha sufrido variaciones significativas. Para ello es necesario convertir la capacidad del sensor en un valor que posteriormente es procesado digitalmente. Por otro lado, con el fin de mejorar la efectividad en el uso de los sensores táctiles 2D desarrollados, se ha estudiado el modo de incorporar un sistema actuador. De esta forma, el usuario recibe una retroalimentación indicando que la orden o acción ha sido reconocida. Para ello, se ha complementado la matriz de sensores capacitivos con una pantalla electroluminiscente también impresa. El resultado final ofrece una solución que combina un sensor táctil 2D con un actuador electroluminiscente realizado mediante impresión electrónica sobre sustrato textil. Posteriormente, se ha llevado a cabo el desarrollo de un sensor de gestos 3D empleando una combinación de sensores impresos también sobre sustrato textil. En este tipo de sensor 3D, se envía una señal que genera un campo eléctrico sobre los sensores impresos. Esto se lleva a cabo mediante un electrodo de transmisión situado muy cerca de ellos. El campo generado es recibido por los sensores y convertido a señales eléctricas. Para ello, los sensores se basan en electrodos que actúan de receptores. Si una persona coloca su mano dentro del área de emisión, se crea una perturbación de las líneas de los campos eléctricos. Esto es debido a la desviación de las líneas de campo a tierra utilizando la conductividad intrínseca del cuerpo humano. Esta perturbación cambia/afecta a las señales recibidas por los electrodos. Las variaciones captadas por todos los electrodos son procesadas de forma conjunta pudiendo determinar la posición y el movimiento de la mano sobre la superficie del sensor. Finalmente, se ha llevado a cabo el desarrollo de un sensor de gestos 3D mejorado. Al igual que el desarrollo anterior, permite la detección de gestos sin necesidad de contacto, pero incrementando la distancia de alcance. Además de la tecnología de impresión electrónica, se ha evaluado el empleo de otras dos tecnologías de fabricación textil.[CA] La present tesi doctoral té com a objectiu fonamental el desenvolupament de nous sensors i actuadors fent servir la tecnologia de electrònica impresa, també coneguda com Printed Electronics. Es va fer us de materials polimèrics conductors, semiconductors i dielèctrics sobre substrats flexibles i/o elàstics. Per mitjà de dissenys i processos d'aplicació adequats, és possible fabricar sensors capaços d'interactuar amb l'entorn. D'aquesta manera, es poden incorporar als substrats, com ara teixits tèxtils, funcionalitats específiques de mesurament de l'entorn i de resposta davant canvis d'aquest. Addicionalment, és necessari incloure sistemes electrònics, capaços de realitzar el processament de les dades obtingudes, així com del seu registre. En el desenvolupament d'aquests sensors i actuadors es combinen les propietats físiques dels diferents materials de forma precisa. Cal dissenyar estructures multicapa on les propietats d'uns materials interaccionen amb les de la resta. manera El resultat es un sensor capaç de captar variacions físiques de l'entorn, i convertirles en senyals que poden ser processades i convertides en dades. D'una banda, s'ha desenvolupat un sensor tàctil imprès sobre substrat tèxtil per a reconeixement de gestos en 2D. Aquest sensor es compon d'una matriu formada amb petits sensors capacitius basats en una estructura de tipus condensador. Aquests s'han dissenyat de manera que, si un dit o un altre objecte amb propietats capacitives s'aproxima prou, el seu comportament varia, podent ser mesurat. Els petits sensors estan ordenats en aquesta matriu com en una quadrícula. Cada sensor té una posició que ve determinada per una fila i per una columna. Periòdicament es mesura la capacitat de cada petit sensor per tal d'avaluar si ha sofert variacions significatives. Per a això cal convertir la capacitat del sensor a un valor que posteriorment és processat digitalment. D'altra banda, per tal de millorar l'efectivitat en l'ús dels sensors tàctils 2D desenvolupats, s'ha estudiat la manera d'incorporar un sistema actuador. D'aquesta forma, l'usuari rep una retroalimentació indicant que l'ordre o acció ha estat reconeguda. Per a això, s'ha complementat la matriu de sensors capacitius amb una pantalla electroluminescent també impresa. El resultat final ofereix una solució que combina un sensor tàctil 2D amb un actuador electroluminescent realitzat mitjançant impressió electrònica sobre substrat tèxtil. Posteriorment, s'ha dut a terme el desenvolupament d'un sensor de gestos 3D emprant una combinació d'un mínim de sensors impresos també sobre substrat tèxtil. En aquest tipus de sensor 3D, s'envia un senyal que genera un camp elèctric sobre els sensors impresos. Això es porta a terme mitjançant un elèctrode de transmissió situat molt a proper a ells. El camp generat és rebut pels sensors i convertit a senyals elèctrics. Per això, els sensors es basen en elèctrodes que actuen de receptors. Si una persona col·loca la seva mà dins de l'àrea d'emissió, es crea una pertorbació de les línies dels camps elèctrics. Això és a causa de la desviació de les línies de camp a terra utilitzant la conductivitat intrínseca de el cos humà. Aquesta pertorbació afecta als senyals rebudes pels elèctrodes. Les variacions captades per tots els elèctrodes són processades de manera conjunta per determinar la posició i el moviment de la mà sobre la superfície del sensor. Finalment, s'ha dut a terme el desenvolupament d'un sensor de gestos 3D millorat. A l'igual que el desenvolupament anterior, permet la detecció de gestos sense necessitat de contacte, però incrementant la distància d'abast. A més a més de la tecnologia d'impressió electrònica, s'ha avaluat emprar altres dues tecnologies de fabricació tèxtil.Ferri Pascual, J. (2020). Tactile and Touchless Sensors Printed on Flexible Textile Substrates for Gesture Recognition [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/153075TESISCompendi

A Wearable Textile 2D Touchpad Sensor Based on Screen-Printing Technology

[EN] Among many of the designs used in the detection of 2D gestures for portable technology, the touchpad is one of the most complex and with more functions to implement. Its development has undergone a great push due to its use in displays, but it is not widely used with other technologies. Its application on textiles could allow a wide range of applications in the field of medicine, sports, etc. Obtaining a flexible, robust touchpad with good response and low cost is one of the objectives of this work. A textile touchpad based on a diamond pattern design using screen printing technology has been developed. This technology is widely used in the textile industry and therefore does not require heavy investments. The developed prototypes were analyzed using a particular controller for projected capacitive technologies (pro-cap), which is the most used in gesture detection. Two different designs were used to obtain the best configuration, obtaining a good result in both cases.This work was supported by Spanish Government/FEDER funds (grant number MAT2015-64139-C4-3-R (Mineco/Feder)). The work presented is also funded by the Conselleria d'Economia Sostenible, Sectors Productius i Treball, through IVACE (Instituto Valenciano de Competitividad Empresarial) and co-funded by ERDF funding from the EU. Application No. IMAMCI/2017/1.Ferri Pascual, J.; Lidon-Roger, JV.; Moreno Canton, J.; Martinez, G.; Garcia-Breijo, E. (2017). A Wearable Textile 2D Touchpad Sensor Based on Screen-Printing Technology. Materials. 10(12):1-16. https://doi.org/10.3390/ma10121450S1161012Takamatsu, S., Lonjaret, T., Ismailova, E., Masuda, A., Itoh, T., & Malliaras, G. G. (2015). Wearable Keyboard Using Conducting Polymer Electrodes on Textiles. Advanced Materials, 28(22), 4485-4488. doi:10.1002/adma.201504249McMillan, D., Brown, B., Lampinen, A., McGregor, M., Hoggan, E., & Pizza, S. (2017). Situating Wearables. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. doi:10.1145/3025453.3025993Nirjon, S., Gummeson, J., Gelb, D., & Kim, K.-H. (2015). TypingRing. Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services - MobiSys ’15. doi:10.1145/2742647.2742665Rekimoto, J. (s. f.). GestureWrist and GesturePad: unobtrusive wearable interaction devices. Proceedings Fifth International Symposium on Wearable Computers. doi:10.1109/iswc.2001.962092Kim, K., Joo, D., & Lee, K.-P. (2010). Wearable-object-based interaction for a mobile audio device. Proceedings of the 28th of the international conference extended abstracts on Human factors in computing systems - CHI EA ’10. doi:10.1145/1753846.1754070Yoon, S. H., Huo, K., & Ramani, K. (2016). Wearable textile input device with multimodal sensing for eyes-free mobile interaction during daily activities. Pervasive and Mobile Computing, 33, 17-31. doi:10.1016/j.pmcj.2016.04.008Van Heek, J., Schaar, A. K., Trevisan, B., Bosowski, P., & Ziefle, M. (2014). User requirements for wearable smart textiles. Does the usage context matter (medical vs. sports)? Proceedings of the 8th International Conference on Pervasive Computing Technologies for Healthcare. doi:10.4108/icst.pervasivehealth.2014.255179Rogers, J. A., Someya, T., & Huang, Y. (2010). Materials and Mechanics for Stretchable Electronics. Science, 327(5973), 1603-1607. doi:10.1126/science.1182383Fan, J. A., Yeo, W.-H., Su, Y., Hattori, Y., Lee, W., Jung, S.-Y., … Rogers, J. A. (2014). Fractal design concepts for stretchable electronics. Nature Communications, 5(1). doi:10.1038/ncomms4266Bhalla, M. R., & Bhalla, A. V. (2010). Comparative Study of Various Touchscreen Technologies. International Journal of Computer Applications, 6(8), 12-18. doi:10.5120/1097-1433Walker, G. (2012). A review of technologies for sensing contact location on the surface of a display. Journal of the Society for Information Display, 20(8), 413-440. doi:10.1002/jsid.100Pedersen, H. C., Jakobsen, M. L., Hanson, S. G., Mosgaard, M., Iversen, T., & Korsgaard, J. (2011). Optical touch screen based on waveguide sensing. Applied Physics Letters, 99(6), 061102. doi:10.1063/1.3615656Emamian, S., Avuthu, S. G. R., Narakathu, B. B., Eshkeiti, A., Chlaihawi, A. A., Bazuin, B. J., … Atashbar, M. Z. (2015). Fully printed and flexible piezoelectric based touch sensitive skin. 2015 IEEE SENSORS. doi:10.1109/icsens.2015.7370651George, B., Zangl, H., Bretterklieber, T., & Brasseur, G. (2010). A Combined Inductive–Capacitive Proximity Sensor for Seat Occupancy Detection. IEEE Transactions on Instrumentation and Measurement, 59(5), 1463-1470. doi:10.1109/tim.2010.2040910Gunnarsson, E., Karlsteen, M., Berglin, L., & Stray, J. (2014). A novel technique for direct measurements of contact resistance between interlaced conductive yarns in a plain weave. Textile Research Journal, 85(5), 499-511. doi:10.1177/0040517514532158Enokibori, Y., Suzuki, A., Mizuno, H., Shimakami, Y., & Mase, K. (2013). E-textile pressure sensor based on conductive fiber and its structure. Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication - UbiComp ’13 Adjunct. doi:10.1145/2494091.2494158Wei, Y., Torah, R., Li, Y., & Tudor, J. (2016). Dispenser printed capacitive proximity sensor on fabric for applications in the creative industries. Sensors and Actuators A: Physical, 247, 239-246. doi:10.1016/j.sna.2016.06.005Gorgutsa, S., Gu, J. F., & Skorobogatiy, M. (2011). A woven 2D touchpad sensor and a 1D slide sensor using soft capacitor fibers. Smart Materials and Structures, 21(1), 015010. doi:10.1088/0964-1726/21/1/015010Hamdan, N. A., Heller, F., Wacharamanotham, C., Thar, J., & Borchers, J. (2016). Grabrics. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’16. doi:10.1145/2851581.2892529Kim, D.-K. (2010). A Touchpad for Force and Location Sensing. ETRI Journal, 32(5), 722-728. doi:10.4218/etrij.10.1510.007

Flexible stretchable electronics for sport and wellbeing applications

Wearable electronics are becoming increasingly widespread in modern society. Though these devices are intended to be worn, integrated into clothing and other everyday objects, the technologies and processes used to manufacture them is no different than those that manufacture laptops and mobile phones. Many of these devices are intended to monitor the user’s health, activity and general wellbeing, within clinical, recreational and assistive environments. Consequently, the inherent incompatibility of these rigid devices with the soft, elastic structure of the human body can in some cases can be uncomfortable and inconvenient for everyday life. For devices to take the step from a ‘wearable’ to an ‘invisible’, a drastic rethinking of electronics manufacturing is required.The fundamental aim of this research is to establish parameters of usefulness and an array of materials with complimentary processes that would assist in transitioning devices to long term almost invisible items that can assist in improving the health of the wearer. In order to approach this problem, a novel architecture was devised that utilised PDMS as a substrate and microfluid channels of Galinstan liquid alloy for interconnects. CO2 laser machining was investigated as a means of creating channels and vias on PDMS substrates. Trace speeds and laser power outputs were investigated in order to find an optimal combination. The results displayed upper limits for power densities; where surpassing this limit resulted in poor repeatability and surface finish. It was found that there was an optimal set of trace speeds that ranged from approximately 120mm/s to 190mm/s that resulted in the most reliable and repeatable performance. Due to the complex nature of a materials variable energy absorption properties, it is not possible to quantify a single optimal parameter set.To understand the performance of these devices in situ, finite element analysis was employed to model deformations that such a device could experience. The aims here were to investigate the bond strength required to prevent delamination, between the silicon-PDMS and PDMS-PDMS bonds, in addition to the stress applied to the silicone die during these deformations. Based upon the applied loads the required bond strengths would need to be at least ~65kPa to maintain PDMS-PDMS adhesion during these tests, while stress on the silicone-PDMS adhesion required an expected v higher ~160kPa, both of which are within the reach of existing bonding techniques that are capable of withstanding a pressure of ~600kPa before failure occurs. Stress on the silicon die did not exceed ~7.8 MPa during simulation, which is well below the fracture stress.By developing knowledge about how various components of such a system will respond during use and under stress, it allows future engineers to make informed design decisions and develop better more resilient products.</div

Blending the Material and Digital World for Hybrid Interfaces

The development of digital technologies in the 21st century is progressing continuously and new device classes such as tablets, smartphones or smartwatches are finding their way into our everyday lives. However, this development also poses problems, as these prevailing touch and gestural interfaces often lack tangibility, take little account of haptic qualities and therefore require full attention from their users. Compared to traditional tools and analog interfaces, the human skills to experience and manipulate material in its natural environment and context remain unexploited. To combine the best of both, a key question is how it is possible to blend the material world and digital world to design and realize novel hybrid interfaces in a meaningful way. Research on Tangible User Interfaces (TUIs) investigates the coupling between physical objects and virtual data. In contrast, hybrid interfaces, which specifically aim to digitally enrich analog artifacts of everyday work, have not yet been sufficiently researched and systematically discussed. Therefore, this doctoral thesis rethinks how user interfaces can provide useful digital functionality while maintaining their physical properties and familiar patterns of use in the real world. However, the development of such hybrid interfaces raises overarching research questions about the design: Which kind of physical interfaces are worth exploring? What type of digital enhancement will improve existing interfaces? How can hybrid interfaces retain their physical properties while enabling new digital functions? What are suitable methods to explore different design? And how to support technology-enthusiast users in prototyping? For a systematic investigation, the thesis builds on a design-oriented, exploratory and iterative development process using digital fabrication methods and novel materials. As a main contribution, four specific research projects are presented that apply and discuss different visual and interactive augmentation principles along real-world applications. The applications range from digitally-enhanced paper, interactive cords over visual watch strap extensions to novel prototyping tools for smart garments. While almost all of them integrate visual feedback and haptic input, none of them are built on rigid, rectangular pixel screens or use standard input modalities, as they all aim to reveal new design approaches. The dissertation shows how valuable it can be to rethink familiar, analog applications while thoughtfully extending them digitally. Finally, this thesis’ extensive work of engineering versatile research platforms is accompanied by overarching conceptual work, user evaluations and technical experiments, as well as literature reviews.Die Durchdringung digitaler Technologien im 21. Jahrhundert schreitet stetig voran und neue Geräteklassen wie Tablets, Smartphones oder Smartwatches erobern unseren Alltag. Diese Entwicklung birgt aber auch Probleme, denn die vorherrschenden berührungsempfindlichen Oberflächen berücksichtigen kaum haptische Qualitäten und erfordern daher die volle Aufmerksamkeit ihrer Nutzer:innen. Im Vergleich zu traditionellen Werkzeugen und analogen Schnittstellen bleiben die menschlichen Fähigkeiten ungenutzt, die Umwelt mit allen Sinnen zu begreifen und wahrzunehmen. Um das Beste aus beiden Welten zu vereinen, stellt sich daher die Frage, wie neuartige hybride Schnittstellen sinnvoll gestaltet und realisiert werden können, um die materielle und die digitale Welt zu verschmelzen. In der Forschung zu Tangible User Interfaces (TUIs) wird die Verbindung zwischen physischen Objekten und virtuellen Daten untersucht. Noch nicht ausreichend erforscht wurden hingegen hybride Schnittstellen, die speziell darauf abzielen, physische Gegenstände des Alltags digital zu erweitern und anhand geeigneter Designparameter und Entwurfsräume systematisch zu untersuchen. In dieser Dissertation wird daher untersucht, wie Materialität und Digitalität nahtlos ineinander übergehen können. Es soll erforscht werden, wie künftige Benutzungsschnittstellen nützliche digitale Funktionen bereitstellen können, ohne ihre physischen Eigenschaften und vertrauten Nutzungsmuster in der realen Welt zu verlieren. Die Entwicklung solcher hybriden Ansätze wirft jedoch übergreifende Forschungsfragen zum Design auf: Welche Arten von physischen Schnittstellen sind es wert, betrachtet zu werden? Welche Art von digitaler Erweiterung verbessert das Bestehende? Wie können hybride Konzepte ihre physischen Eigenschaften beibehalten und gleichzeitig neue digitale Funktionen ermöglichen? Was sind geeignete Methoden, um verschiedene Designs zu erforschen? Wie kann man Technologiebegeisterte bei der Erstellung von Prototypen unterstützen? Für eine systematische Untersuchung stützt sich die Arbeit auf einen designorientierten, explorativen und iterativen Entwicklungsprozess unter Verwendung digitaler Fabrikationsmethoden und neuartiger Materialien. Im Hauptteil werden vier Forschungsprojekte vorgestellt, die verschiedene visuelle und interaktive Prinzipien entlang realer Anwendungen diskutieren. Die Szenarien reichen von digital angereichertem Papier, interaktiven Kordeln über visuelle Erweiterungen von Uhrarmbändern bis hin zu neuartigen Prototyping-Tools für intelligente Kleidungsstücke. Um neue Designansätze aufzuzeigen, integrieren nahezu alle visuelles Feedback und haptische Eingaben, um Alternativen zu Standard-Eingabemodalitäten auf starren Pixelbildschirmen zu schaffen. Die Dissertation hat gezeigt, wie wertvoll es sein kann, bekannte, analoge Anwendungen zu überdenken und sie dabei gleichzeitig mit Bedacht digital zu erweitern. Dabei umfasst die vorliegende Arbeit sowohl realisierte technische Forschungsplattformen als auch übergreifende konzeptionelle Arbeiten, Nutzerstudien und technische Experimente sowie die Analyse existierender Forschungsarbeiten