Enabling Expressive Keyboard Interaction with Finger, Hand, and Hand Posture Identification

The input space of conventional physical keyboards is largely limited by the number of keys. To enable more actions than simply entering the symbol represented by a key, standard keyboards use combinations of modifier keys such as command, alternate, or shift to re-purpose the standard text entry behaviour. To explore alternatives to conventional keyboard shortcuts and enable more expressive keyboard interaction, this thesis first presents Finger-Aware Shortcuts, which encode information from finger, hand, and hand posture identification as keyboard shortcuts. By detecting the hand and finger used to press a key, and an open or closed hand posture, a key press can have multiple command mappings. A formative study revealed the performance and preference patterns when using different fingers and postures to press a key. The results were used to develop a computer vision algorithm to identify fingers and hands on a keyboard captured by a built-in laptop camera and a reflector. This algorithm was built into a background service to enable system-wide Finger-Aware Shortcut keys in any application. A controlled experiment used the service to compare the performance of Finger-Aware Shortcuts with existing methods. The results showed that Finger-Aware Shortcuts are comparable with a common class of shortcuts using multiple modifier keys. Several application demonstrations illustrate different use cases and mappings for Finger-Aware Shortcuts. To further explore how introducing finger awareness can help foster the learning and use of keyboard shortcuts, an interview study was conducted with expert computer users to identify the likely causes that hinder the adoption of keyboard shortcuts. Based on this, the concept of Finger-Aware Shortcuts is extended and two guided keyboard shortcut techniques are proposed: FingerArc and FingerChord. The two techniques provide dynamic visual guidance on the screen when users press and hold an alphabetical key semantically related to a set of commands. FingerArc differentiates these commands by examining the angle between the thumb and index finger; FingerChord differentiates these commands by allowing users to press different key areas using a second finger. The thesis contributes comprehensive evaluations of Finger-Aware Shortcuts and proof-of-concept demonstrations of FingerArc and FingerChord. Together, they contribute a novel interaction space that expands the conventional keyboard input space with more expressivity

Ubiquitous computing and natural interfaces for environmental information

Dissertação apresentada na Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa para obtenção do Grau de Mestre em Engenharia do Ambiente, perfil Gestão e Sistemas AmbientaisThe next computing revolution‘s objective is to embed every street, building, room and object with computational power. Ubiquitous computing (ubicomp) will allow every object to receive and transmit information, sense its surroundings and act accordingly, be located from anywhere in the world, connect every person. Everyone will have the possibility to access information, despite their age, computer knowledge, literacy or physical impairment. It will impact the world in a profound way, empowering mankind, improving the environment, but will also create new challenges that our society, economy, health and global environment will have to overcome. Negative impacts have to be identified and dealt with in advance. Despite these concerns, environmental studies have been mostly absent from discussions on the new paradigm. This thesis seeks to examine ubiquitous computing, its technological emergence, raise awareness towards future impacts and explore the design of new interfaces and rich interaction modes. Environmental information is approached as an area which may greatly benefit from ubicomp as a way to gather, treat and disseminate it, simultaneously complying with the Aarhus convention. In an educational context, new media are poised to revolutionize the way we perceive, learn and interact with environmental information. cUbiq is presented as a natural interface to access that information

RotoSwype : word-gesture typing using a ring

Funding: NSERC Discovery Grant #2018-05187, the Canada Foundation for Innovation Infrastructure Fund “Facility for Fully Interactive Physio-digital Spaces” (#33151), and Ontario Early Researcher Award #ER16-12-184.We propose RotoSwype, a technique for word-gesture typing using the orientation of a ring worn on the index finger. RotoSwype enables one-handed text-input without encumbering the hand with a device, a desirable quality in many scenarios, including virtual or augmented reality. The method is evaluated using two arm positions: with the hand raised up with the palm parallel to the ground; and with the hand resting at the side with the palm facing the body. A five-day study finds both hand positions achieved speeds of at least 14 words-per-minute (WPM) with uncorrected error rates near 1%, outperforming previous comparable techniques.Postprin

TouchEditor: Interaction design and evaluation of a flexible touchpad for text editing of head-mounted displays in speech-unfriendly environments

A text editing solution that adapts to speech-unfriendly (inconvenient to speak or difficult to recognize speech) environments is essential for head-mounted displays (HMDs) to work universally. For existing schemes, e.g., touch bar, virtual keyboard and physical keyboard, there are shortcomings such as insufficient speed, uncomfortable experience or restrictions on user location and posture. To mitigate these restrictions, we propose TouchEditor, a novel text editing system for HMDs based on a flexible piezoresistive film sensor, supporting cursor positioning, text selection, text retyping and editing commands (i.e., Copy, Paste, Delete, etc.). Through literature overview and heuristic study, we design a pressure-controlled menu and a shortcut gesture set for entering editing commands, and propose an area-and-pressure-based method for cursor positioning and text selection that skillfully maps gestures in different areas and with different strengths to cursor movements with different directions and granularities. The evaluation results show that TouchEditor i) adapts to various contents and scenes well with a stable correction speed of 0.075 corrections per second; ii) achieves 95.4% gesture recognition accuracy; iii) reaches a considerable level with a mobile phone in text selection tasks. The comparison results with the speech-dependent EYEditor and the built-in touch bar further prove the flexibility and robustness of TouchEditor in speech-unfriendly environments

Pinching sweaters on your phone – iShoogle : multi-gesture touchscreen fabric simulator using natural on-fabric gestures to communicate textile qualities

The inability to touch fabrics online frustrates consumers, who are used to evaluating physical textiles by engaging in complex, natural gestural interactions. When customers interact with physical fabrics, they combine cross-modal information about the fabric's look, sound and handle to build an impression of its physical qualities. But whenever an interaction with a fabric is limited (i.e. when watching clothes online) there is a perceptual gap between the fabric qualities perceived digitally and the actual fabric qualities that a person would perceive when interacting with the physical fabric. The goal of this thesis was to create a fabric simulator that minimized this perceptual gap, enabling accurate perception of the qualities of fabrics presented digitally. We designed iShoogle, a multi-gesture touch-screen sound-enabled fabric simulator that aimed to create an accurate representation of fabric qualities without the need for touching the physical fabric swatch. iShoogle uses on-screen gestures (inspired by natural on-fabric movements e.g. Crunching) to control pre-recorded videos and audio of fabrics being deformed (e.g. being Crunched). iShoogle creates an illusion of direct video manipulation and also direct manipulation of the displayed fabric. This thesis describes the results of nine studies leading towards the development and evaluation of iShoogle. In the first three studies, we combined expert and non-expert textile-descriptive words and grouped them into eight dimensions labelled with terms Crisp, Hard, Soft, Textured, Flexible, Furry, Rough and Smooth. These terms were used to rate fabric qualities throughout the thesis. We observed natural on-fabric gestures during a fabric handling study (Study 4) and used the results to design iShoogle's on-screen gestures. In Study 5 we examined iShoogle's performance and speed in a fabric handling task and in Study 6 we investigated users' preferences for sound playback interactivity. iShoogle's accuracy was then evaluated in the last three studies by comparing participants’ ratings of textile qualities when using iShoogle with ratings produced when handling physical swatches. We also described the recording and processing techniques for the video and audio content that iShoogle used. Finally, we described the iShoogle iPhone app that was released to the general public. Our evaluation studies showed that iShoogle significantly improved the accuracy of fabric perception in at least some cases. Further research could investigate which fabric qualities and which fabrics are particularly suited to be represented with iShoogle

Roughness experiments of viscous fluid interfaces in disordered Hele-Shaw cells

[spa] Se presenta un trabajo experimental sobre el crecimiento de interficies rugosas en medios desordenados. El experimento consiste en una celda de Hele-Shaw en cuya placa inferior se han distribuido una serie de obstáculos de cobre que hacen la función de desorden congelado. Un aceite de silicona es inyectado por uno de los extremos de la celda desplazando el aire inicialmente presente. El frente es estable por contraste de viscosidad pero se hace rugoso debido a la presencia del desorden. Los experimentos realizados se pueden clasificar en dos grupos principales según el tipo de mecanismo utilizado para la inyección del aceite: A- Velocidad constante y B- Presión constante. Para el Caso A- se han realizado multitud de experimentos variando la velocidad del fluido, el espaciado entre placas en la celda de Hele-Shaw y la configuración del desorden. Se ha observado que cuando la velocidad promedio del aceite es alta y/o la longitud de correlación del desorden en la dirección de crecimiento es baja (desorden débil), entonces el escalamiento de las fluctuaciones de la interfície sigue el escalamiento dinámico de Family-Vicsek, con un exponente de crecimiento de beta=0,50 independiente de la velocidad promedio de la interface, espaciado entre placas y configuración del desorden. También se ha estudiado del exponente de rugosidad, encontrando dos regímenes, alfa-1 a escalas pequeñas y alfa-2 a escalas grandes, con unos valores que dependen de la velocidad promedio, el espaciado entre placas y la configuración del desorden. Para velocidades grandes se obtienen un valor límite de alfa 1=1,3 y alfa 2=0 independientemente de los parámetros experimentales. También se ha caracterizado el punto de crossover entre alfa-1 y alfa-2, el cual escala con la velocidad con un exponente 0.5, de acuerdo con predicciones teóricas

NASA Tech Briefs, December 1991

Topics include: Electronic Components and Circuits. Electronic Systems, Physical Sciences, Materials, Computer Programs, Mechanics, Machinery, Fabrication Technology, Mathematics and Information Sciences

Metafore mobilnih komunikacija ; Метафоры мобильной связи.

Mobilne komunikacije su polje informacione i komunikacione tehnologije koje karakteriše brzi razvoj i u kome se istraživanjem u analitičkim okvirima kognitivne lingvistike, zasnovanom na uzorku od 1005 odrednica, otkriva izrazito prisustvo metafore, metonimije, analogije i pojmovnog objedinjavanja. Analiza uzorka reči i izraza iz oblasti mobilnih medija, mobilnih operativnih sistema, dizajna korisničkih interfejsa, terminologije mobilnih mreža, kao i slenga i tekstizama koje upotrebljavaju korisnici mobilnih naprava ukazuje da pomenuti kognitivni mehanizmi imaju ključnu ulogu u olakšavanju interakcije između ljudi i širokog spektra mobilnih uređaja sa računarskim sposobnostima, od prenosivih računara i ličnih digitalnih asistenata (PDA), do mobilnih telefona, tableta i sprava koje se nose na telu. Ti mehanizmi predstavljaju temelj razumevanja i nalaze se u osnovi principa funkcionisanja grafičkih korisničkih interfejsa i direktne manipulacije u računarskim okruženjima. Takođe je analiziran i poseban uzorak od 660 emotikona i emođija koji pokazuju potencijal za proširenje značenja, imajući u vidu značaj piktograma za tekstualnu komunikaciju u vidu SMS poruka i razmenu tekstualnih sadržaja na društvenim mrežama kojima se redovno pristupa putem mobilnih uređaja...Mobile communications are a fast-developing field of information and communication technology whose exploration within the analytical framework of cognitive linguistics, based on a sample of 1005 entries, reveals the pervasive presence of metaphor, metonymy analogy and conceptual integration. The analysis of the sample consisting of words and phrases related to mobile media, mobile operating systems and interface design, the terminology of mobile networking, as well as the slang and textisms employed by mobile gadget users shows that the above cognitive mechanisms play a key role in facilitating interaction between people and a wide range of mobile computing devices from laptops and PDAs to mobile phones, tablets and wearables. They are the cornerstones of comprehension that are behind the principles of functioning of graphical user interfaces and direct manipulation in computing environments. A separate sample, featuring a selection of 660 emoticons and emoji, exhibiting the potential for semantic expansion was also analyzed, in view of the significance of pictograms for text-based communication in the form of text messages or exchanges on social media sites regularly accessed via mobile devices..

Roughness experiments of viscous fluid interfaces in disordered Hele-Shaw cells

Se presenta un trabajo experimental sobre el crecimiento de interficies rugosas en medios desordenados. El experimento consiste en una celda de Hele-Shaw en cuya placa inferior se han distribuido una serie de obstáculos de cobre que hacen la función de desorden congelado. Un aceite de silicona es inyectado por uno de los extremos de la celda desplazando el aire inicialmente presente. El frente es estable por contraste de viscosidad pero se hace rugoso debido a la presencia del desorden. Los experimentos realizados se pueden clasificar en dos grupos principales según el tipo de mecanismo utilizado para la inyección del aceite: A- Velocidad constante y B- Presión constante. Para el Caso A- se han realizado multitud de experimentos variando la velocidad del fluido, el espaciado entre placas en la celda de Hele-Shaw y la configuración del desorden. Se ha observado que cuando la velocidad promedio del aceite es alta y/o la longitud de correlación del desorden en la dirección de crecimiento es baja (desorden débil), entonces el escalamiento de las fluctuaciones de la interfície sigue el escalamiento dinámico de Family-Vicsek, con un exponente de crecimiento de beta=0,50 independiente de la velocidad promedio de la interface, espaciado entre placas y configuración del desorden. También se ha estudiado del exponente de rugosidad, encontrando dos regímenes, alfa-1 a escalas pequeñas y alfa-2 a escalas grandes, con unos valores que dependen de la velocidad promedio, el espaciado entre placas y la configuración del desorden. Para velocidades grandes se obtienen un valor límite de alfa 1=1,3 y alfa 2=0 independientemente de los parámetros experimentales. También se ha caracterizado el punto de crossover entre alfa-1 y alfa-2, el cual escala con la velocidad con un exponente 0.5, de acuerdo con predicciones teóricas