Crossmodal audio and tactile interaction with mobile touchscreens

Touchscreen mobile devices often use cut-down versions of desktop user interfaces placing high demands on the visual sense that may prove awkward in mobile settings. The research in this thesis addresses the problems encountered by situationally impaired mobile users by using crossmodal interaction to exploit the abundant similarities between the audio and tactile modalities. By making information available to both senses, users can receive the information in the most suitable way, without having to abandon their primary task to look at the device. This thesis begins with a literature review of related work followed by a definition of crossmodal icons. Two icons may be considered to be crossmodal if and only if they provide a common representation of data, which is accessible interchangeably via different modalities. Two experiments investigated possible parameters for use in crossmodal icons with results showing that rhythm, texture and spatial location are effective. A third experiment focused on learning multi-dimensional crossmodal icons and the extent to which this learning transfers between modalities. The results showed identification rates of 92% for three-dimensional audio crossmodal icons when trained in the tactile equivalents, and identification rates of 89% for tactile crossmodal icons when trained in the audio equivalent. Crossmodal icons were then incorporated into a mobile touchscreen QWERTY keyboard. Experiments showed that keyboards with audio or tactile feedback produce fewer errors and greater speeds of text entry compared to standard touchscreen keyboards. The next study examined how environmental variables affect user performance with the same keyboard. The data showed that each modality performs differently with varying levels of background noise or vibration and the exact levels at which these performance decreases occur were established. The final study involved a longitudinal evaluation of a touchscreen application, CrossTrainer, focusing on longitudinal effects on performance with audio and tactile feedback, the impact of context on performance and personal modality preference. The results show that crossmodal audio and tactile icons are a valid method of presenting information to situationally impaired mobile touchscreen users with recognitions rates of 100% over time. This thesis concludes with a set of guidelines on the design and application of crossmodal audio and tactile feedback to enable application and interface designers to employ such feedback in all systems

Reflection 5

Society for Cultural Interaction in East Asia, The 1st General Assembly and 1st Annual Meeting 2009 New Approaches to Multicultural Interaction, Column 1: Who Owns World Heritage?– An Attempt for Sharing Historical Understanding, ICIS Periphery Project The 2nd Practice-based Fieldwork in Suburban Settlements of the Old Outer Port of Hue, Vietnam, Activities 1: International Symposiums and Meetings Educational Situation in ICIS, Column 2: Cultural Interaction Studies on Food (4), Activities 2 : Faculty Seminars, Publications / Personnel Changes, Solicitation of Submissions for the Bulletin of ICIS /Editer’s Not

Reflection 4

The First International Academic Forum for the Next Generation Cultural Reproduction on its Interface : From the Perspectives of Text, Diplomacy, Image of Others, and Tea Culture in East Asia, Column1: Cultural Interaction Studies from the perspective of the Corn Field, The Third ICIS International Forum Chinese Culture Viewed from the Peripheral Perspective, Activities, Column 2: Cultural Interaction Studies on Food （3）, The First Conversation of RAs in Cultural Interaction Studies Program, New Publication, Announcements, Solicitation of Submissions for the Bulletin of ICIS / Editor’s Not

Reflection 8

The Third International Forum for Next Generation、The Establishment of a New Department for Research on East Asian Culture : As a New Research Base for the Study on Cultural Interaction Studies、Report on the Field Work in Amakusa、International Symposium、Faculty Seminars、Column1／Mystery of “Dutch Water”、Column2／Cultural Interaction Studies on Food（7）、Activity Reports and Announcements、Editor’s Note

VEMI Lab 2021

The Virtual Environments and Multimodal Interaction (VEMI) Lab embodies an inclusive, collaborative, and multi-disciplinary approach to hands-on research and education. By bringing together students and faculty from more than a dozen majors and disciplines, VEMI is uniquely positioned to advance computing and STEM initiatives both here at the university as well as in broader communities throughout Maine and nationwide

Reflection 1

Message from the President, Opening Ceremony Speech, Institute Overview, Organizational Chart, Membership, Some Preliminary Observations on the History of Chinese and Japanese Cultural Interaction, First International Symposium Considering the Potential of Cultural Interaction Studies, Activity Report, Notices and Editor’s Not

Reflection 9

The Report on Fieldwork in Amakusa，2011ICIS International Symposium“Rethinking Vietnam, Korea, and Ryukyu in East Asia from the View of Periphery and Centre”3rd Annual Meeting of Society for Cultural Interaction in East AsiaInternational Symposium on the Activity Reports of RA/Reports on the participation of ASCJ and WHAThe 5th International SymposiumColumn: “Border Crossing” by GodsActivity ReportsSolicitation of Submissions for the Bulletin / Editor’s Not

Reflection 7

The Report on Fieldwork in Amakusa, The 5th ICIS International Workshop, The 4th International Symposium, International Academic Symposium, Cultural Interaction Studies on Food (6), Activities and Announcements, Solicitation of Submissions for the Bulletin of ICIS / Editor\u27s Not