Crossmodal audio and tactile interaction with mobile touchscreens

Touchscreen mobile devices often use cut-down versions of desktop user interfaces placing high demands on the visual sense that may prove awkward in mobile settings. The research in this thesis addresses the problems encountered by situationally impaired mobile users by using crossmodal interaction to exploit the abundant similarities between the audio and tactile modalities. By making information available to both senses, users can receive the information in the most suitable way, without having to abandon their primary task to look at the device. This thesis begins with a literature review of related work followed by a definition of crossmodal icons. Two icons may be considered to be crossmodal if and only if they provide a common representation of data, which is accessible interchangeably via different modalities. Two experiments investigated possible parameters for use in crossmodal icons with results showing that rhythm, texture and spatial location are effective. A third experiment focused on learning multi-dimensional crossmodal icons and the extent to which this learning transfers between modalities. The results showed identification rates of 92% for three-dimensional audio crossmodal icons when trained in the tactile equivalents, and identification rates of 89% for tactile crossmodal icons when trained in the audio equivalent. Crossmodal icons were then incorporated into a mobile touchscreen QWERTY keyboard. Experiments showed that keyboards with audio or tactile feedback produce fewer errors and greater speeds of text entry compared to standard touchscreen keyboards. The next study examined how environmental variables affect user performance with the same keyboard. The data showed that each modality performs differently with varying levels of background noise or vibration and the exact levels at which these performance decreases occur were established. The final study involved a longitudinal evaluation of a touchscreen application, CrossTrainer, focusing on longitudinal effects on performance with audio and tactile feedback, the impact of context on performance and personal modality preference. The results show that crossmodal audio and tactile icons are a valid method of presenting information to situationally impaired mobile touchscreen users with recognitions rates of 100% over time. This thesis concludes with a set of guidelines on the design and application of crossmodal audio and tactile feedback to enable application and interface designers to employ such feedback in all systems

A move-step analysis of the concluding chapters in computer science phd theses

[EN] This paper describes how computer science doctoral writers construct the closing chapters of their PhD theses. The data are drawn from the chapters playing a concluding role of 48 PhD theses defended at the University of Glasgow from 2008 to 2014. The analysis applied a qualitative-quantitative approach. The titles of the concluding chapters of the theses were first examined and also their divisions into sections and sub-sections. Then the chapters were subjected to a move-step analysis: Move 1 (M1) “Revisiting the study”; Move 2 (M2) “Consolidating research space”; Move 3 (M3) “Proposing practical applications and implications”, Move 4 (M4) “Recommending future work” and Move 5 (M5) “Recapitulating the study”. The results revealed that most of the computer science PhD theses have one final concluding chapter with three main moves: M1, M2 and M4. The most frequent steps are “reviewing the work carried out” and “summarizing the specific work reported in every thesis chapter” in M1, “presenting results and contributions”, “answering the initial research questions or hypotheses”, and “making claims” in M2, and “acknowledging limitations” and “suggesting further research” in M4. Movestep patterns appear in recurrent cycles throughout the concluding chapters. Several suggestions for pedagogical purposes are provided.[ES] Este artículo describe cómo los autores de tesis doctorales en el área de la informática elaboran los capítulos de conclusión. Los datos proceden de los capítulos finales de 48 tesis doctorales defendidas en la Universidad de Glasgow entre 2008 y 2014. Para el análisis se siguió un enfoque cualitativo y cuantitativo. En una primera etapa, se examinaron los títulos de los capítulos de conclusión de las tesis así como sus divisiones en secciones y subsecciones. Posteriormente, se analizaron los capítulos atendiendo a unidades informativas organizadas en movimientos y pasos: Movimiento 1 (M1) “Revisión del estudio”; Movimiento 2 (M2) “Consolidación del espacio de investigación”; Movimiento 3 (M3) “Propuesta de aplicaciones prácticas e implicaciones”, Movimiento 4 (M4) “Recomendaciones para futuras investigaciones” y Movimiento 5 (M5) “Recapitulación del estudio”. Los resultados indican que la mayoría de las tesis de este corpus de informática tiene un único capítulo final de conclusiones con tres movimientos principales: M1, M2 y M4. Los pasos más frecuentes consisten en “revisar el trabajo llevado a cabo” y “resumir el trabajo específico desarrollado en cada capítulo de la tesis” en M1, “enunciar los resultados y las contribuciones en respuesta a las hipótesis y preguntas iniciales” y “reivindicar su aportación” en M2, y “reconocer limitaciones de la investigación” y “sugerir investigaciones futuras” en M4. La práctica habitual consiste en utilizar patrones de movimientos y pasos en ciclos recurrentes a lo largo de la conclusión. Se aportan sugerencias con fines pedagógicos.Soler Monreal, C. (2016). A move-step analysis of the concluding chapters in computer science phd theses. Ibérica. (32):105-132. http://hdl.handle.net/10251/80709S1051323

Rhetorical strategies in PhD conclusions of computer science: From the review of the study to consolidation of research space

[ES] Este artículo analiza los patrones de movimientos que predominan en los capítulos finales de conclusión de 48 tesis doctorales de informática en una universidad británica. Se centra en la naturaleza y frecuencia de las conexiones entre pasos del Movimiento 1 sobre la revisión del trabajo de investigación y los pasos del Movimiento 2, de consolidación del espacio investigador. Las combinaciones más comunes relacionan (1) el resumen del trabajo de la tesis con el producto y su evaluación, (2) el propósito y la hipótesis inicial con los resultados, (3) las preguntas de investigación con la metodología, el producto y la reivindicación, (4) un problema o necesidad con una metodología específica, un nuevo producto y/o una reivindicación, y (5) un resumen del trabajo realizado en cada capítulo con los resultados y reivindicaciones. Algunos de los resultados obtenidos son específicos del área de la informática. Del estudio se desprenden implicaciones pedagógicas para cursos de inglés para fines específicos (IFA).[EN] This study investigates the predominant moves and move patterns used in the separate final conclusion chapters of 48 PhD theses of computer science at a UK university. The focus is on the most salient connections of steps in the review of the study (Move 1) with steps for the consolidation of research space (Move 2). The most common combinations relate (1) a summary of the thesis work to the product and the evaluation of the product, (2) the purpose, thesis statement or hypothesis to the findings or results, (3) the research questions to the methodology, product or claim, (4) a problem or need to a specific methodology, a new product and/or a claim, and (5) a summary of the work done in each thesis chapter to the findings and claims. Some findings are specific of the field of computer science. The study has pedagogical implications for courses of English for Academic Purposes (EAP).Soler Monreal, C. (2019). Rhetorical strategies in PhD conclusions of computer science: From the review of the study to consolidation of research space. Revista Española de Lingüística Aplicada/Spanish Journal of Applied Linguistics. 32(1):356-384. https://doi.org/10.1075/resla.16034.solS356384321Badley, G. (2009). Academic writing as shaping and re-shaping. Teaching in Higher Education, 14(2), 209-219. doi:10.1080/13562510902757294Basturkmen, H. (2009). Commenting on results in published research articles and masters dissertations in Language Teaching. Journal of English for Academic Purposes, 8(4), 241-251. doi:10.1016/j.jeap.2009.07.001Basturkmen, H. (2012). A genre-based investigation of discussion sections of research articles in Dentistry and disciplinary variation. Journal of English for Academic Purposes, 11(2), 134-144. doi:10.1016/j.jeap.2011.10.004Bitchener, J., & Basturkmen, H. (2006). Perceptions of the difficulties of postgraduate L2 thesis students writing the discussion section. Journal of English for Academic Purposes, 5(1), 4-18. doi:10.1016/j.jeap.2005.10.002Bunton, D. (2005). The structure of PhD conclusion chapters. Journal of English for Academic Purposes, 4(3), 207-224. doi:10.1016/j.jeap.2005.03.004Cotos, E. (2014). Genre-Based Automated Writing Evaluation for L2 Research Writing. doi:10.1057/9781137333377Cumming, A., Lai, C., & Cho, H. (2016). Students’ writing from sources for academic purposes: A synthesis of recent research. Journal of English for Academic Purposes, 23, 47-58. doi:10.1016/j.jeap.2016.06.002Geng, Y., & Wharton, S. (2016). Evaluative language in discussion sections of doctoral theses: Similarities and differences between L1 Chinese and L1 English writers. Journal of English for Academic Purposes, 22, 80-91. doi:10.1016/j.jeap.2016.01.001Holmes, R. (1997). Genre analysis, and the social sciences: An investigation of the structure of research article discussion sections in three disciplines. English for Specific Purposes, 16(4), 321-337. doi:10.1016/s0889-4906(96)00038-5Holmes, R. (2001). Variation and Text Structure. ITL - International Journal of Applied Linguistics, 131-132, 107-137. doi:10.1075/itl.131-132.06holJohns, A. M., & Swales, J. M. (2002). Literacy and disciplinary practices: opening and closing perspectives. Journal of English for Academic Purposes, 1(1), 13-28. doi:10.1016/s1475-1585(02)00003-6Kuteeva, M., & Negretti, R. (2016). Graduate students’ genre knowledge and perceived disciplinary practices: Creating a research space across disciplines. English for Specific Purposes, 41, 36-49. doi:10.1016/j.esp.2015.08.004Kwan, B. S. C. (2006). The schematic structure of literature reviews in doctoral theses of applied linguistics. English for Specific Purposes, 25(1), 30-55. doi:10.1016/j.esp.2005.06.001Lim, J. M.-H. (2014). Formulating research questions in experimental doctoral dissertations on Applied Linguistics. English for Specific Purposes, 35, 66-88. doi:10.1016/j.esp.2014.02.003Lim, J. M.-H., Loi, C.-K., Hashim, A., & Liu, M. S.-M. (2015). Purpose statements in experimental doctoral dissertations submitted to U.S. universities: An inquiry into doctoral students’ communicative resources in language education. Journal of English for Academic Purposes, 20, 69-89. doi:10.1016/j.jeap.2015.06.002Paltridge, B. (2002). Thesis and dissertation writing: an examination of published advice and actual practice. English for Specific Purposes, 21(2), 125-143. doi:10.1016/s0889-4906(00)00025-9Paltridge, B., Starfield, S., Ravelli, L. J., & Tuckwell, K. (2012). Change and stability: Examining the macrostructures of doctoral theses in the visual and performing arts. Journal of English for Academic Purposes, 11(4), 332-344. doi:10.1016/j.jeap.2012.08.003Parkinson, J. (2011). The Discussion section as argument: The language used to prove knowledge claims. English for Specific Purposes, 30(3), 164-175. doi:10.1016/j.esp.2011.03.001Peacock, M. (2002). Communicative moves in the discussion section of research articles. System, 30(4), 479-497. doi:10.1016/s0346-251x(02)00050-7Posteguillo, S. (1999). The Schematic Structure of Computer Science Research Articles. English for Specific Purposes, 18(2), 139-160. doi:10.1016/s0889-4906(98)00001-5Samraj, B. (2008). A discourse analysis of master’s theses across disciplines with a focus on introductions. Journal of English for Academic Purposes, 7(1), 55-67. doi:10.1016/j.jeap.2008.02.005Soler-Monreal, C. (2015). Announcing one’s work in PhD theses in computer science: A comparison of Move 3 in literature reviews written in English L1, English L2 and Spanish L1. English for Specific Purposes, 40, 27-41. doi:10.1016/j.esp.2015.07.004Soler-Monreal, C., Carbonell-Olivares, M., & Gil-Salom, L. (2011). A contrastive study of the rhetorical organisation of English and Spanish PhD thesis introductions. English for Specific Purposes, 30(1), 4-17. doi:10.1016/j.esp.2010.04.005Swales, J. M. (2004). Research Genres. doi:10.1017/cbo9781139524827Swales, J., & Feak, C. (2000). English in Today’s Research World. doi:10.3998/mpub.9059Ruiying, Y., & Allison, D. (2003). Research articles in applied linguistics: moving from results to conclusions. English for Specific Purposes, 22(4), 365-385. doi:10.1016/s0889-4906(02)00026-1Yayli, D. (2011). From genre awareness to cross-genre awareness: A study in an EFL context. Journal of English for Academic Purposes, 10(3), 121-129. doi:10.1016/j.jeap.2011.02.00

Investigating perceptual congruence between information and sensory parameters in auditory and vibrotactile displays

A fundamental interaction between a computer and its user(s) is the transmission of information between the two and there are many situations where it is necessary for this interaction to occur non-visually, such as using sound or vibration. To design successful interactions in these modalities, it is necessary to understand how users perceive mappings between information and acoustic or vibration parameters, so that these parameters can be designed such that they are perceived as congruent. This thesis investigates several data-sound and data-vibration mappings by using psychophysical scaling to understand how users perceive the mappings. It also investigates the impact that using these methods during design has when they are integrated into an auditory or vibrotactile display. To investigate acoustic parameters that may provide more perceptually congruent data-sound mappings, Experiments 1 and 2 explored several psychoacoustic parameters for use in a mapping. These studies found that applying amplitude modulation — or roughness — to a signal, or applying broadband noise to it resulted in performance which were similar to conducting the task visually. Experiments 3 and 4 used scaling methods to map how a user perceived a change in an information parameter, for a given change in an acoustic or vibrotactile parameter. Experiment 3 showed that increases in acoustic parameters that are generally considered undesirable in music were perceived as congruent with information parameters with negative valence such as stress or danger. Experiment 4 found that data-vibration mappings were more generalised — a given increase in a vibrotactile parameter was almost always perceived as an increase in an information parameter — regardless of the valence of the information parameter. Experiments 5 and 6 investigated the impact that using results from the scaling methods used in Experiments 3 and 4 had on users' performance when using an auditory or vibrotactile display. These experiments also explored the impact that the complexity of the context which the display was placed had on user performance. These studies found that using mappings based on scaling results did not significantly impact user's performance with a simple auditory display, but it did reduce response times in a more complex use-case

Remote tactile feedback on interactive surfaces

Direct touch input on interactive surfaces has become a predominating standard for the manipulation of digital information in our everyday lives. However, compared to our rich interchange with the physical world, the interaction with touch-based systems is limited in terms of flexibility of input and expressiveness of output. Particularly, the lack of tactile feedback greatly reduces the general usability of a touch-based system and hinders from a productive entanglement of the virtual information with the physical world. This thesis proposes remote tactile feedback as a novel method to provide programmed tactile stimuli supporting direct touch interactions. The overall principle is to spatially decouple the location of touch input (e.g. fingertip or hand) and the location of the tactile sensation on the user's body (e.g. forearm or back). Remote tactile feedback is an alternative concept which avoids particular challenges of existing approaches. Moreover, the principle provides inherent characteristics which can accommodate for the requirements of current and future touch interfaces. To define the design space, the thesis provides a structured overview of current forms of touch surfaces and identifies trends towards non-planar and non-rigid forms with more versatile input mechanisms. Furthermore, a classification highlights limitations of the current methods to generate tactile feedback on touch-based systems. The proposed notion of tactile sensory relocation is a form of sensory substitution. Underlying neurological and psychological principles corroborate the approach. Thus, characteristics of the human sense of touch and principles from sensory substitution help to create a technical and conceptual framework for remote tactile feedback. Three consecutive user studies measure and compare the effects of both direct and remote tactile feedback on the performance and the subjective ratings of the user. Furthermore, the experiments investigate different body locations for the application of tactile stimuli. The results show high subjective preferences for tactile feedback, regardless of its type of application. Additionally, the data reveals no significant differences between the effects of direct and remote stimuli. The results back the feasibility of the approach and provide parameters for the design of stimuli and the effective use of the concept. The main part of the thesis describes the systematical exploration and analysis of the inherent characteristics of remote tactile feedback. Four specific features of the principle are identified: (1) the simplification of the integration of cutaneous stimuli, (2) the transmission of proactive, reactive and detached feedback, (3) the increased expressiveness of tactile sensations and (4) the provision of tactile feedback during multi-touch. In each class, several prototypical remote tactile interfaces are used in evaluations to analyze the concept. For example, the PhantomStation utilizes psychophysical phenomena to reduce the number of single tactile actuators. An evaluation with the prototype compares standard actuator technologies with each other in order to enable simple and scalable implementations. The ThermalTouch prototype creates remote thermal stimuli to reproduce material characteristics on standard touchscreens. The results show a stable rate of virtual object discrimination based on remotely applied temperature profiles. The AutmotiveRTF system is implemented in a vehicle and supports the driver's input on the in-vehicle-infotainment system. A field study with the system focuses on evaluating the effects of proactive and reactive feedback on the user's performance. The main contributions of the dissertation are: First, the thesis introduces the principle of remote tactile feedback and defines a design space for this approach as an alternative method to provide non-visual cues on interactive surfaces. Second, the thesis describes technical examples to rapidly prototype remote tactile feedback systems. Third, these prototypes are deployed in several evaluations which highlight the beneficial subjective and objective effects of the approach. Finally, the thesis presents features and inherent characteristics of remote tactile feedback as a means to support the interaction on today's touchscreens and future interactive surfaces.Die Interaktion mit berührungsempfindlichen Oberflächen ist heute ein Standard für die Manipulation von digitaler Information. Jedoch weist die Bedienung dieser interaktiven Bildschirme starke Einschränkungen hinsichtlich der Flexibilität bei der Eingabe und der Ausdruckskraft der Ausgabe auf, wenn man sie mit den vielfältigen Möglichkeiten des Umgangs mit Objekten in unserer Alltagswelt vergleicht. Besonders die nicht vorhandenen Tastsinnesrückmeldungen vermindern stark die Benutzbarkeit solcher Systeme und verhindern eine effektive Verknüpfung von virtueller Information und physischer Welt. Die vorliegende Dissertation beschreibt den Ansatz der 'distalen taktilen Rückmeldungen' als neuartige Möglichkeit zur Vermittlung programmierter Tastsinnesreize an Benutzer interaktiver Oberflächen. Das Grundprinzip dabei ist die räumliche Trennung zwischen der Eingabe durch Berührung (z.B. mit der Fingerspitze) und dem daraus resultierenden taktilen Reiz am Körper der Benutzer (z.B. am Rücken). Dabei vermeidet das Konzept der distalen taktilen Rückmeldungen einzelne technische und konzeptionelle Nachteile existierender Ansätze. Zusätzlich bringt es Interaktionsmöglichkeiten mit sich, die den Eigenheiten der Interaktion mit aktuellen und auch zukünftigen berührungsempfindlichen Oberflächen Rechnung tragen. Zu Beginn zeigt ein Überblick zu relevanten Arbeiten den aktuellen Forschungstrend hin zu nicht-flachen und verformbaren berührungsempfindlichen Oberflächen sowie zu vielfältigeren Eingabemethoden. Eine Klassifizierung ordnet existierende technische Verfahren zur Erzeugung von künstlichen Tastsinnesreizen und stellt jeweils konzeptuelle und technische Herausforderungen dar. Der in dieser Arbeit vorgeschlagene Ansatz der Verlagerung von Tastsinnesreizen ist eine Form der sensorischen Substitution, zugrunde liegende neurologische und psychologische Prinzipien untermauern das Vorgehen. Die Wirkprinzipien des menschlichen Tastsinnes und die Systeme zur sensorischen Substitution liefern daher konzeptionelle und technische Richtlinien zur Umsetzung der distalen taktilen Rückmeldungen. Drei aufeinander aufbauende Benutzerstudien vergleichen die Auswirkungen von direkten und distalen taktilen Rückmeldungen auf die Leistung und das Verhalten von Benutzern sowie deren subjektive Bewertung der Interaktion. Außerdem werden in den Experimenten die Effekte von Tastsinnesreizen an verschiedenen Körperstellen untersucht. Die Ergebnisse zeigen starke Präferenzen für Tastsinnesrückmeldungen, unabhängig von deren Applikationsort. Die Daten ergeben weiterhin keine signifikanten Unterschiede bei den quantitativen Effekten von direktem und distalen Rückmeldungen. Diese Ergebnisse befürworten die Realisierbarkeit des Ansatzes und zeigen Richtlinien für weitere praktische Umsetzungen auf. Der Hauptteil der Dissertation beschreibt die systematische Untersuchung und Analyse der inhärenten Möglichkeiten, die sich aus der Vermittlung distaler taktiler Rückmeldungen ergeben. Vier verschiedene Charakteristika werden identifiziert: (1) die vereinfachte Integration von Tastsinnesreizen, (2) die Vermittlung von proaktiven, reaktiven und entkoppelten Rückmeldungen, (3) die erhöhte Bandbreite der taktilen Signale und (4) die Darstellung von individuellen Tastsinnesreizen für verschiedene Kontaktpunkte mit der berührungsempfindlichen Oberfläche. Jedes dieser Prinzipien wird durch prototypische Systeme umgesetzt und in Benutzerstudien analysiert. Beispielsweise nutzt das System PhantomStation psychophysikalische Illusionen, um die Anzahl der einzelnen Reizgeber zu reduzieren. In einer Evaluierung des Prototypen werden mehrere Aktuatortechnologien verglichen, um einfache und skalierbare Ansätze zu identifizieren. Der ThermalTouch-Prototyp wird dazu genutzt, distale thermale Reize zu vermitteln, um so Materialeigenschaften auf Berührungsbildschirmen darstellen zu können. Eine Benutzerstudie zeigt, dass sich auf Basis dieser Temperaturverläufe virtuelle Objekte unterscheiden lassen. Das AutomotiveRTF-System wird schließlich in ein Kraftfahrzeug integriert, um den Fahrer bei der Eingabe auf dem Informations- und Unterhaltungssystem zu unterstützen. Eine Feldstudie untersucht die Auswirkungen der proaktiven und reaktiven Rückmeldungen auf die Benutzerleistung. Die vorliegende Dissertation leistet mehrere Beiträge zur Mensch-Maschine-Interaktion: Das Prinzip der distalen taktilen Rückmeldungen wird eingeführt als Alternative zur Erzeugung nicht-visueller Rückmeldungen auf interaktiven Oberflächen. Es werden technische Verfahrensweisen zur prototypischen Implementierung solcher Systeme vorgeschlagen. Diese technischen Prototypen werden in einer Vielzahl verschiedener Benutzerstudien eingesetzt, welche die quantitativen und qualitativen Vorteile des Ansatzes aufzeigen. Schließlich wird gezeigt, wie sich das Prinzip zur Unterstützung heutiger und zukünftiger Interaktionsformen mit berührungsempfindlichen Bildschirmen nutzen lässt