Wearable Vibrotactile Haptic Device for Stiffness Discrimination during Virtual Interactions

In this paper, we discuss the development of cost effective, wireless, and wearable vibrotactile haptic device for stiffness perception during an interaction with virtual objects. Our experimental setup consists of haptic device with five vibrotactile actuators, virtual reality environment tailored in Unity 3D integrating the Oculus Rift Head Mounted Display (HMD) and the Leap Motion controller. The virtual environment is able to capture touch inputs from users. Interaction forces are then rendered at 500 Hz and fed back to the wearable setup stimulating fingertips with ERM vibrotactile actuators. Amplitude and frequency of vibrations are modulated proportionally to the interaction force to simulate the stiffness of a virtual object. A quantitative and qualitative study is done to compare the discrimination of stiffness on virtual linear spring in three sensory modalities: visual only feedback, tactile only feedback, and their combination. A common psychophysics method called the Two Alternative Forced Choice (2AFC) approach is used for quantitative analysis using Just Noticeable Difference (JND) and Weber Fractions (WF). According to the psychometric experiment result, average Weber fraction values of 0.39 for visual only feedback was improved to 0.25 by adding the tactile feedback

Analysis of different vibration patterns to guide blind people

[EN] The literature indicates the best vibration positions and frequencies on the human body where tactile information is transmitted. However, there is a lack of knowledge about how to combine tactile stimuli for navigation. The aim of this study is to compare different vibration patterns outputted to blind people and to determine the most intuitive vibration patterns to indicate direction for navigation purposes through a tactile belt. The vibration patterns that stimulate the front side of the waist are preferred for indicating direction. Vibration patterns applied on the back side of the waist could be suitable for sending messages such as stop.This research is supported by Seventh European Framework Programme (FP7/2007-2013) under grant agreement 605821 (BLINDTRACK). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.Durá-Gil, JV.; Bazuelo-Ruiz, B.; Moro Pérez, D.; Molla Domenech, F. (2017). Analysis of different vibration patterns to guide blind people. PeerJ. 5:1-10. https://doi.org/10.7717/peerj.30821105Cholewiak, R. W. (1999). The Perception of Tactile Distance: Influences of Body Site, Space, and Time. Perception, 28(7), 851-875. doi:10.1068/p2873Cholewiak, R. W., Brill, J. C., & Schwab, A. (2004). Vibrotactile localization on the abdomen: Effects of place and space. Perception & Psychophysics, 66(6), 970-987. doi:10.3758/bf03194989Cholewiak, R. W., & Craig, J. C. (1984). Vibrotactile pattern recognition and discrimination at several body sites. Perception & Psychophysics, 35(6), 503-514. doi:10.3758/bf03205946Faugloire, E., & Lejeune, L. (2014). Evaluation of heading performance with vibrotactile guidance: The benefits of information–movement coupling compared with spatial language. Journal of Experimental Psychology: Applied, 20(4), 397-410. doi:10.1037/xap0000032Ghiani, G., Leporini, B., & Paternò, F. (2009). Vibrotactile feedback to aid blind users of mobile guides. Journal of Visual Languages & Computing, 20(5), 305-317. doi:10.1016/j.jvlc.2009.07.004Johnson, L. A., & Higgins, C. M. (2006). A Navigation Aid for the Blind Using Tactile-Visual Sensory Substitution. 2006 International Conference of the IEEE Engineering in Medicine and Biology Society. doi:10.1109/iembs.2006.259473Kärcher, S. M., Fenzlaff, S., Hartmann, D., Nagel, S. K., & König, P. (2012). Sensory Augmentation for the Blind. Frontiers in Human Neuroscience, 6. doi:10.3389/fnhum.2012.00037Marston, J. R., Loomis, J. M., Klatzky, R. L., Golledge, R. G., & Smith, E. L. (2006). Evaluation of spatial displays for navigation without sight. ACM Transactions on Applied Perception, 3(2), 110-124. doi:10.1145/1141897.1141900Ranjbar, P., & Stenström, I. (2013). Monitor, a Vibrotactile Aid for Environmental Perception: A Field Evaluation by Four People with Severe Hearing and Vision Impairment. The Scientific World Journal, 2013, 1-11. doi:10.1155/2013/206734Saaty, T. L. (1990). How to make a decision: The analytic hierarchy process. European Journal of Operational Research, 48(1), 9-26. doi:10.1016/0377-2217(90)90057-iSrikulwong, M., & O’Neill, E. (2010). A Comparison of Two Wearable Tactile Interfaces with a Complementary Display in Two Orientations. Lecture Notes in Computer Science, 139-148. doi:10.1007/978-3-642-15841-4_15Van Erp, J. B. F. (2008). Absolute localization of vibrotactile stimuli on the torso. Perception & Psychophysics, 70(6), 1016-1023. doi:10.3758/pp.70.6.101

Crossmodal audio and tactile interaction with mobile touchscreens

Touchscreen mobile devices often use cut-down versions of desktop user interfaces placing high demands on the visual sense that may prove awkward in mobile settings. The research in this thesis addresses the problems encountered by situationally impaired mobile users by using crossmodal interaction to exploit the abundant similarities between the audio and tactile modalities. By making information available to both senses, users can receive the information in the most suitable way, without having to abandon their primary task to look at the device. This thesis begins with a literature review of related work followed by a definition of crossmodal icons. Two icons may be considered to be crossmodal if and only if they provide a common representation of data, which is accessible interchangeably via different modalities. Two experiments investigated possible parameters for use in crossmodal icons with results showing that rhythm, texture and spatial location are effective. A third experiment focused on learning multi-dimensional crossmodal icons and the extent to which this learning transfers between modalities. The results showed identification rates of 92% for three-dimensional audio crossmodal icons when trained in the tactile equivalents, and identification rates of 89% for tactile crossmodal icons when trained in the audio equivalent. Crossmodal icons were then incorporated into a mobile touchscreen QWERTY keyboard. Experiments showed that keyboards with audio or tactile feedback produce fewer errors and greater speeds of text entry compared to standard touchscreen keyboards. The next study examined how environmental variables affect user performance with the same keyboard. The data showed that each modality performs differently with varying levels of background noise or vibration and the exact levels at which these performance decreases occur were established. The final study involved a longitudinal evaluation of a touchscreen application, CrossTrainer, focusing on longitudinal effects on performance with audio and tactile feedback, the impact of context on performance and personal modality preference. The results show that crossmodal audio and tactile icons are a valid method of presenting information to situationally impaired mobile touchscreen users with recognitions rates of 100% over time. This thesis concludes with a set of guidelines on the design and application of crossmodal audio and tactile feedback to enable application and interface designers to employ such feedback in all systems

BIOPHOTONICS IN BIOPHARMACEUTICAL APPLICATIONS

Biopharmaceutical products are composed of complex or ordinate combinations of proteins, lipids, sugars, and nucleic acids or living cells or tissues. Due to the intrinsic variability of biological systems and the complexity of the bioprocesses involved in the production of these products, new technologies are required to monitor and characterize, the production processes and the final products. Biophotonic techniques, particularly Raman and Infrared (IR) spectroscopy are rapid, robust, operator independent, non-destructive and label free, thus particularly suitable for these purposes. This dissertation first investigates the use of biophotonic techniques in research of Extracellular Vesicles (EVs). EVs act as intercellular messengers and therefore have considerable potential in drug delivery system, diagnostic biomarker, or therapeutic agent. Subsequently highlighting the need and potential of a new kind of time gated Raman spectrometer to be created. Raman and IR spectroscopy are methods able to characterize and assess the quality of EV suspensions with different degree of purity. These vibrational spectroscopic techniques show different intrinsic advantages, as they are label free and operator independent methods. Particularly, Raman might be the most suitable technology since it is less sensitive to water compared to IR. Raman spectroscopy reveals the information on the chemical composition and physical status of the analyte. However, it does not provide information of the analyte environment which the spectrofluorometer can gather. EVs studied by spectrofluorometer are required to be labelled with a fluorescent dye. Results obtained by fluorescence lifetime imaging spectroscopy underline that the cell uptake of the fluorescently labelled EVs is feasible. Attention must be paid to the efficacy of the labelling and further to the elimination of the unbound dye since the labelling may severely compromise the results or lead to wrong conclusions on EV functionality. The combined advantage of Raman spectroscopy and fluorescence decay are obtained by a previously in-house developed time resolved Raman spectrometer. Thanks to the peculiar sensor of the spectrometer, the width of the time gate can be modified which is used to separate the Raman signal from the fluorescence tail. The modifications can be done even in the data post-processing phase, to obtain the best possible Raman signal-to-noise ratios. The simultaneously detected Raman spectra and time-resolved fluorescence decay curves are used to study the diffusion of small molecular drugs in a hydrogel. The data reveal the chemical composition, physical status, and the interaction with the environment of the samples. Taken together the obtained results suggest that the quality of EV suspensions can be assessed by Raman spectroscopy and their cell uptake detected by fluorescence lifetime spectroscopy. Both Raman spectra and fluorescence decay can be measured simultaneously by a second generation of time-resolved Raman spectrometer.Biofarmaseuttiset tuotteet koostuvat proteiinien, lipidien, sokereiden ja nukleiinihappojen tai elävien solujen tai kudosten järjestäytyneistä tai monimutkaisista yhdistelmistä. Biologisten järjestelmien luontaisen vaihtelevuuden ja näiden tuotteiden valmistukseen liittyvien bioprosessien monimutkaisuuden vuoksi tarvitaan uusia teknologioita tuotantoprosessien ja lopputuotteiden seurantaa ja karakterisointia varten. Biofotoniset tekniikat, erityisesti Raman- ja Infrapuna (IR) -spektroskopia ovat nopeita, luotettavia, hellävaraisia, leimattomia ja käyttäjästä riippumattomia tekniikoita, jotka soveltuvat erityisen hyvin biofarmaseuttisten tuotteiden määritykseen. Tämä väitöskirjan alussa tutkitaan biofotonisten tekniikoiden käyttöä solunulkoisten vesikkeleiden (EV:iden) tutkimuksessa. EV:t toimivat solujenvälisessä viestinnässä ja siksi niillä on huomattava potentiaali lääkekehityksessä, diagnostisina biomarkkerina ja terapeuttisina tekijöinä. Myöhemmin tutkimuksessa korostetaan uudenlaisen aikaerotteisen Raman-spektrometrin tarvetta ja mahdollisuuksia biofarmaseuttisten tuotteiden määrityksessä. Raman- ja IR-spektroskopia ovat menetelmiä, joilla voidaan karakterisoida ja arvioida eri puhtausasteen omaavia EV-näytteiden laatua. Nämä värähtelyspektroskooppiset tekniikat ovat leimattomia ja käyttäjästä riippumattomia menetelmiä. Verrattuna IR-spektroskopiaan, Raman on luonteeltaan mahdollisesti käyttökelpoisin tekniikka EV:iden karakterisointiin, sillä veden läsnäolo näytteessä ei häiritse mittausta. Raman-spektroskopian avulla saadaan tietoa analyytin kemiallisesta koostumuksesta sekä fysikaalisesta tilasta ja spektrofluorometrin avulla saadaan kerättyä informaatiota analyytin ympäristöstä, mihin Raman ei puolestaan pysty. Spektrofluorometrillä tutkitut EV:t on leimattava fluoresoivalla väriaineella ja fluoresenssin eliniän kuvantamisspektroskopialla saadut tulokset korostavat, että fluoresenssileimattujen EV:iden soluunotto on mahdollista. Leimauksen tehokkuuteen ja sitoutumattoman väriaineen eliminointiin on kiinnitettävä huomiota, sillä virheellinen leimaus voi väärentää tuloksia tai johtaa virheellisiin johtopäätöksiin EV:iden funktionaalisuudesta. Talon sisällä rakennetun laitteiston avulla on saatu yhdistettyä Raman-spektroskopia ja fluoresenssin eliniän vaimeneminen. Spektrometrin omalaatuisen anturin ansiosta kuvantamisen aikaikkunaa pystytään muokkaamaan, minkä avulla Raman-signaali pystytään erottamaan fluoresenssisignaalista. Muutokset voidaan tehdä myös datan jälkikäsittelyvaiheessa parhaan mahdollisen Raman-signaali-kohinasuhteen saamiseksi. Pienmolekyylisten lääkkeiden diffuusion tutkimiseen hydrogeelissä voidaan käyttää samanaikaisesti havaittuja Raman-spektrejä ja aikaerotteisia fluoresenssin vaimenemiskäyriä. Kerätty aineisto paljastaa näytteiden kemiallisen koostumuksen, fysikaalisen tilan ja vuorovaikutuksen ympäristön kanssa. Yhdessä saadut tulokset viittaavat siihen, että EV-suspensioiden laatua voidaan arvioida Raman-spektroskopialla ja niiden soluunotto havaita fluoresenssielinaikaspektroskopialla, kun Raman-spektri ja fluoresenssin heikkeneminen voidaan mitata samanaikaisesti toisen sukupolven aikaerotteisella Raman-spektrometrillä

Somatic ABC's: A Theoretical Framework for Designing, Developing and Evaluating the Building Blocks of Touch-Based Information Delivery

abstract: Situations of sensory overload are steadily becoming more frequent as the ubiquity of technology approaches reality--particularly with the advent of socio-communicative smartphone applications, and pervasive, high speed wireless networks. Although the ease of accessing information has improved our communication effectiveness and efficiency, our visual and auditory modalities--those modalities that today's computerized devices and displays largely engage--have become overloaded, creating possibilities for distractions, delays and high cognitive load; which in turn can lead to a loss of situational awareness, increasing chances for life threatening situations such as texting while driving. Surprisingly, alternative modalities for information delivery have seen little exploration. Touch, in particular, is a promising candidate given that it is our largest sensory organ with impressive spatial and temporal acuity. Although some approaches have been proposed for touch-based information delivery, they are not without limitations including high learning curves, limited applicability and/or limited expression. This is largely due to the lack of a versatile, comprehensive design theory--specifically, a theory that addresses the design of touch-based building blocks for expandable, efficient, rich and robust touch languages that are easy to learn and use. Moreover, beyond design, there is a lack of implementation and evaluation theories for such languages. To overcome these limitations, a unified, theoretical framework, inspired by natural, spoken language, is proposed called Somatic ABC's for Articulating (designing), Building (developing) and Confirming (evaluating) touch-based languages. To evaluate the usefulness of Somatic ABC's, its design, implementation and evaluation theories were applied to create communication languages for two very unique application areas: audio described movies and motor learning. These applications were chosen as they presented opportunities for complementing communication by offloading information, typically conveyed visually and/or aurally, to the skin. For both studies, it was found that Somatic ABC's aided the design, development and evaluation of rich somatic languages with distinct and natural communication units.Dissertation/ThesisPh.D. Computer Science 201

Safe Local Navigation for Visually Impaired Users With a Time-of-Flight and Haptic Feedback Device

This paper presents ALVU (Array of Lidars and Vibrotactile Units), a contactless, intuitive, hands-free, and discreet wearable device that allows visually impaired users to detect low- and high-hanging obstacles, as well as physical boundaries in their immediate environment. The solution allows for safe local navigation in both confined and open spaces by enabling the user to distinguish free space from obstacles. The device presented is composed of two parts: a sensor belt and a haptic strap. The sensor belt is an array of time-of-flight distance sensors worn around the front of a user's waist, and the pulses of infrared light provide reliable and accurate measurements of the distances between the user and surrounding obstacles or surfaces. The haptic strap communicates the measured distances through an array of vibratory motors worn around the user's upper abdomen, providing haptic feedback. The linear vibration motors are combined with a point-loaded pretensioned applicator to transmit isolated vibrations to the user. We validated the device's capability in an extensive user study entailing 162 trials with 12 blind users. Users wearing the device successfully walked through hallways, avoided obstacles, and detected staircases.Andrea Bocelli FoundationNational Science Foundation (U.S.) (Grant NSF IIS1226883