TESSA - Toolkit for experimentation with multimodal sensory substitution and augmentation

TESSA is a toolkit for experimenting with sensory augmentation. It includes hardware and software to facilitate rapid prototyping of interfaces that can enhance one sense using information gathered from another sense. The toolkit contains a range of sensors (e.g. ultrasonics, temperature sensors) and actuators (e.g. tactors or stereo sound), designed modularly so that inputs and outputs can be easily swapped in and out and customized using TESSA’s graphical user interface (GUI), with “real time” feedback. The system runs on a Raspberry Pi with a built-in touchscreen, providing a compact and portable form that is amenable for field trials. At CHI Interactivity, the audience will have the opportunity to experience sensory augmentation effects using this system, and design their own sensory augmentation interfaces

Personalising Vibrotactile Displays through Perceptual Sensitivity Adjustment

Haptic displays are commonly limited to transmitting a discrete set of tactile motives. In this paper, we explore the transmission of real-valued information through vibrotactile displays. We simulate spatial continuity with three perceptual models commonly used to create phantom sensations: the linear, logarithmic and power model. We show that these generic models lead to limited decoding precision, and propose a method for model personalization adjusting to idiosyncratic and spatial variations in perceptual sensitivity. We evaluate this approach using two haptic display layouts: circular, worn around the wrist and the upper arm, and straight, worn along the forearm. Results of a user study measuring continuous value decoding precision show that users were able to decode continuous values with relatively high accuracy (4.4% mean error), circular layouts performed particularly well, and personalisation through sensitivity adjustment increased decoding precision

Somatic ABC's: A Theoretical Framework for Designing, Developing and Evaluating the Building Blocks of Touch-Based Information Delivery

abstract: Situations of sensory overload are steadily becoming more frequent as the ubiquity of technology approaches reality--particularly with the advent of socio-communicative smartphone applications, and pervasive, high speed wireless networks. Although the ease of accessing information has improved our communication effectiveness and efficiency, our visual and auditory modalities--those modalities that today's computerized devices and displays largely engage--have become overloaded, creating possibilities for distractions, delays and high cognitive load; which in turn can lead to a loss of situational awareness, increasing chances for life threatening situations such as texting while driving. Surprisingly, alternative modalities for information delivery have seen little exploration. Touch, in particular, is a promising candidate given that it is our largest sensory organ with impressive spatial and temporal acuity. Although some approaches have been proposed for touch-based information delivery, they are not without limitations including high learning curves, limited applicability and/or limited expression. This is largely due to the lack of a versatile, comprehensive design theory--specifically, a theory that addresses the design of touch-based building blocks for expandable, efficient, rich and robust touch languages that are easy to learn and use. Moreover, beyond design, there is a lack of implementation and evaluation theories for such languages. To overcome these limitations, a unified, theoretical framework, inspired by natural, spoken language, is proposed called Somatic ABC's for Articulating (designing), Building (developing) and Confirming (evaluating) touch-based languages. To evaluate the usefulness of Somatic ABC's, its design, implementation and evaluation theories were applied to create communication languages for two very unique application areas: audio described movies and motor learning. These applications were chosen as they presented opportunities for complementing communication by offloading information, typically conveyed visually and/or aurally, to the skin. For both studies, it was found that Somatic ABC's aided the design, development and evaluation of rich somatic languages with distinct and natural communication units.Dissertation/ThesisPh.D. Computer Science 201

Haptic Socks

Smart phones have been spreading faster than any other technologies and with them, the research for finding different alternate interaction techniques that will be quick and efficiently responsive. The best feedback response in terms of mobile usage has been a long debate and is actually not possible, so it is difficult to conclude which feedback will be effective in all environments. In navigation applications, two modalities are used as feedback; visual and auditory. This thesis presents work, experiments and results on implementing the third modality i.e. haptic feedback. The basic purpose of this work is to find out how effective wearable haptic feedback can be, than visual or auditory feedback in terms of navigation. Using hand-held GPS navigation while walking or driving a car can sometimes be dangerous if the user is focusing more on the device than on the roads. The concept of Haptic Socks can be used as a secondary interaction technique for navigation so that user can use other interaction techniques to perform their primary tasks or perform their daily life routine work. Haptic socks will consist of actuators embedded in a certain position of human foot that will give tactile feedback, helping the user in turn by turn navigation. The device can most probably be the user’s smartphone. Haptic Socks will use wireless connection with the device which in this research study will be Bluetooth. Furthermore, if the feedback results are positive then it will be easier to discuss how effective it can be made for people who are deaf-blind

INTERACTION BETWEEN SIGNAL COMPLEXITY AND PHYSICAL ACTIVITY IN VIBRO-TACTILE COMMUNICATION

Master'sMASTER OF ART

温熱快適性向上と視触覚刺激を用いる作業負荷の低減による運動時パフォーマンス向上に関する研究

筑波大学 (University of Tsukuba)201

SWORD: um dispositivo vibratório vestível para uma rabilitação mais eficiente em doentes com AVC

Doutoramento em Engenharia ElectrotécnicaAnualmente ocorrem cerca de 16 milhões AVCs em todo o mundo. Cerca de metade dos sobreviventes irá apresentar défice motor que necessitará de reabilitação na janela dos 3 aos 6 meses depois do AVC. Nos países desenvolvidos, é estimado que os custos com AVCs representem cerca de 0.27% do Produto Interno Bruto de cada País. Esta situação implica um enorme peso social e financeiro. Paradoxalmente a esta situação, é aceite na comunidade médica a necessidade de serviços de reabilitação motora mais intensivos e centrados no doente. Na revisão do estado da arte, demonstra-se o arquétipo que relaciona metodologias terapêuticas mais intensivas com uma mais proficiente reabilitação motora do doente. Revelam-se também as falhas nas soluções tecnológicas existentes que apresentam uma elevada complexidade e custo associado de aquisição e manutenção. Desta forma, a pergunta que suporta o trabalho de doutoramento seguido inquire a possibilidade de criar um novo dispositivo de simples utilização e de baixo custo, capaz de apoiar uma recuperação motora mais eficiente de um doente após AVC, aliando intensidade com determinação da correcção dos movimentos realizados relativamente aos prescritos. Propondo o uso do estímulo vibratório como uma ferramenta proprioceptiva de intervenção terapêutica a usar no novo dispositivo, demonstra-se a tolerabilidade a este tipo de estímulos através do teste duma primeira versão do sistema apenas com a componente de estimulação num primeiro grupo de 5 doentes. Esta fase validará o subsequente desenvolvimento do sistema SWORD. Projectando o sistema SWORD como uma ferramenta complementar que integra as componentes de avaliação motora e intervenção proprioceptiva por estimulação, é descrito o desenvolvimento da componente de quantificação de movimento que o integra. São apresentadas as diversas soluções estudadas e o algoritmo que representa a implementação final baseada na fusão sensorial das medidas provenientes de três sensores: acelerómetro, giroscópio e magnetómetro. O teste ao sistema SWORD, quando comparado com o método de reabilitação tradicional, mostrou um ganho considerável de intensidade e qualidade na execução motora para 4 dos 5 doentes testados num segundo grupo experimental. É mostrada a versatilidade do sistema SWORD através do desenvolvimento do módulo de Tele-Reabilitação que complementa a componente de quantificação de movimento com uma interface gráfica de feedback e uma ferramenta de análise remota da evolução motora do doente. Finalmente, a partir da componente de quantificação de movimento, foi ainda desenvolvida uma versão para avaliação motora automatizada, implementada a partir da escala WMFT, que visa retirar o factor subjectivo da avaliação humana presente nas escalas de avaliação motora usadas em Neurologia. Esta versão do sistema foi testada num terceiro grupo experimental de cinco doentes.About 16 million first ever-strokes occur worldwide every year. Half of stroke survivors are left with some degree of physical impairment that needs rehabilitation in the 3 to 6 month after-stroke time window. This situation implies a high economic and social burden. In developed countries, stroke cost is estimated to represent an average of 0.27% of each country’s gross domestic product. Paradoxically, it is accepted in the medical community the need for more intensive and patient-centered rehabilitation services. In the state of art review, it is demonstrated the archetype that relates the intensity on rehabilitation with a proficient motor recovery of the patient. Additionally, it is shown that the major pitfalls in current technological solutions in the field of motor rehabilitation are due to their intrinsic complexity and associated cost. Given this state of the art, the research question that supports this thesis, inquiries the possibility of creating a novel low-cost device targeted at the motor rehabilitation of stroke patients, capable of providing a more efficient treatment through enabling higher intensity and automated determination of the correctness of the movements performed by the recovering patient. The validity of the vibratory stimulus is presented from an historic and neurophysiologic point of view. Furthermore, a state of art review of motion capture systems is presented. Intending the use of the vibratory stimulus as a proprioceptive therapeutic tool to be integrated in the new device, it is demonstrated the tolerability of the stimulus from the experimental test of a first version of the device, incorporating the stimulation component, in a first group of five patients. Projecting the SWORD device as a tool that combines both features of motor function evaluation with proprioceptive intervention through vibratory stimulation, it is described the development of the motion capture component. Several solutions were studied and the final algorithm, based on the sensory fusion of the measures from three sensors (accelerometer, gyroscope and magnetometer), is described in detail. The experimental test of the SWORD system on a second group of patients showed that, when compared with a typical treatment, it is capable of providing a more intensive intervention and with a higher quality in 4 out of 5 patients. To demonstrate the versatility of the SWORD system, it was developed the tele-rehabilitation module that complements the motion capture component with a graphical feedback interface and a remote tool for the clinician to evaluate the performance of the patient through out the time he uses the system in his home or any other remote environment. Finally, from the motion capture component, a motor function evaluation version of the system was deployed. Implemented from the WMFT scale, it aims to eliminate the human subjectivity present in the traditional evaluation scales used in the neurology medical area. This system was evaluated on a third group of five patients

Evacuation dynamics in the maritime field: modelling, simulation and real-time human participation

The topic of evacuation analysis is becoming increasingly important in the maritime field, especially after the recent approval of relevant amendments to SOLAS. These amendments make evacuation analysis in early design stage mandatory not only for ro-ro passenger ships, as in the past, but also for other passenger ships, constructed on or after 1st January 2020, carrying more than 36 passengers. Tools used to perform evacuation simulations are generally run in a non-interactive batch mode. However, the introduction of the possibility for humans to interactively participate in a simulated evacuation process together with computer controlled agents in an immersive virtual environment, can open a series of interesting possibilities for design, research and development. Therefore, with particular reference to the maritime field, the research described in this dissertation is focused on the development and implementation of a mathematical model for simulating the dynamics of evacuation processes, which also allows real time human interaction through the use of virtual reality. The developed mathematical model, which is capable of naturally embedding human interaction, was verified and validated through a series of tests and through comparisons with other models and experimental data, as well as by referring to the relevant guidelines proposed by the International Maritime Organization (IMO). Particular attention was given to the calibration and validation of the counterflow model, developed during the research activity, and to the analysis of flow-density relation. The possibility of real time user participation, consisting in the user taking control over an agent inside the simulation, was introduced along with a vibrotactile haptic interface which was created to enhance the user perception of the surrounding virtual environment. The developed tool and user interfaces were adopted in an experiment where the subject was immersed in a virtual environment and interacted with simulated agents. The analysis of experiments provided results on the effects of the developed haptic interface on the subjects\u2019 behaviour. Moreover, the obtained data allowed comparing the behaviour of subjects with that of simulated agents. The mathematical model was subsequently extended with the introduction of ship motion effects on agents behaviour, considering that, in the maritime field, the platform is usually moving. Fictitious forces, in the developed model, are directly applied to the agents and might therefore modify their trajectories. This represents an added value of the proposed model, because, usually, the effects of ship motions are embedded in simulation models only through a speed reduction. The model was used to assess ship motion effects in some IMO test cases. Finally, the tool was tested on a specifically developed case targeting the maritime field whose geometry was ideated as a simplification of the general plan of a real cruise vessel. The evacuation simulations were run firstly without ship motions, then with some representative situations combining heel, trim and periodic motions and, finally, with motions due to irregular waves. Ship motions, in this latter case, have been generated considering a notational cruise vessel whose dimensions were in line with the cruise vessel the test geometry was inspired to. A model introducing ship motion effects on the control of the avatar was finally developed, together with an approach to provide perception of ship motions through the developed vibrotactile interface. Models and results presented in this dissertation provide new insight to the field of ship evacuation analysis and to the application of virtual reality in this field