1,510 research outputs found
Anchorage: Visual Analysis of Satisfaction in Customer Service Videos via Anchor Events
Delivering customer services through video communications has brought new
opportunities to analyze customer satisfaction for quality management. However,
due to the lack of reliable self-reported responses, service providers are
troubled by the inadequate estimation of customer services and the tedious
investigation into multimodal video recordings. We introduce Anchorage, a
visual analytics system to evaluate customer satisfaction by summarizing
multimodal behavioral features in customer service videos and revealing
abnormal operations in the service process. We leverage the semantically
meaningful operations to introduce structured event understanding into videos
which help service providers quickly navigate to events of their interest.
Anchorage supports a comprehensive evaluation of customer satisfaction from the
service and operation levels and efficient analysis of customer behavioral
dynamics via multifaceted visualization views. We extensively evaluate
Anchorage through a case study and a carefully-designed user study. The results
demonstrate its effectiveness and usability in assessing customer satisfaction
using customer service videos. We found that introducing event contexts in
assessing customer satisfaction can enhance its performance without
compromising annotation precision. Our approach can be adapted in situations
where unlabelled and unstructured videos are collected along with sequential
records.Comment: 13 pages. A preprint version of a publication at IEEE Transactions on
Visualization and Computer Graphics (TVCG), 202
Scalable Exploration of Complex Objects and Environments Beyond Plain Visual Replicationâ
Digital multimedia content and presentation means are rapidly increasing their sophistication and are now capable of describing detailed representations of the physical world. 3D exploration experiences allow people to appreciate, understand and interact with intrinsically virtual objects.
Communicating information on objects requires the ability to explore them under different angles, as well as to mix highly photorealistic or illustrative presentations of the object themselves with additional data that provides additional insights on these objects, typically represented in the form of annotations. Effectively providing these capabilities requires the solution of important problems in visualization and user interaction.
In this thesis, I studied these problems in the cultural heritage-computing-domain, focusing on the very common and important special case of mostly planar, but visually, geometrically, and semantically rich objects. These could be generally roughly flat objects with a standard frontal viewing direction (e.g., paintings, inscriptions, bas-reliefs), as well as visualizations of fully 3D objects from a particular point of views (e.g., canonical views of buildings or statues). Selecting a precise application domain and a specific presentation mode allowed me to concentrate on the well defined use-case of the exploration of annotated relightable stratigraphic models (in particular, for local and remote museum presentation).
My main results and contributions to the state of the art have been a novel technique for interactively controlling visualization lenses while automatically maintaining good focus-and-context parameters, a novel approach for avoiding clutter in an annotated model and for guiding users towards interesting areas, and a method for structuring audio-visual object annotations into a graph and for using that graph to improve guidance and support storytelling and automated tours.
We demonstrated the effectiveness and potential of our techniques by performing interactive exploration sessions on various screen sizes and types ranging from desktop devices to large-screen displays for a walk-up-and-use museum installation.
KEYWORDS - Computer Graphics, Human-Computer Interaction, Interactive Lenses, Focus-and-Context, Annotated Models, Cultural Heritage Computing
Video Summarization Using Unsupervised Deep Learning
In this thesis, we address the task of video summarization using unsupervised deep-learning architectures. Video summarization aims to generate a short summary by selecting the most informative and important frames (key-frames) or fragments (key-fragments) of the full-length video, and presenting them in temporally-ordered fashion. Our objective is to overcome observed weaknesses of existing video summarization approaches that utilize RNNs for modeling the temporal dependence of frames, related to: i) the small influence of the estimated frame-level importance scores in the created video summary, ii) the insufficiency of RNNs to model long-range frames' dependence, and iii) the small amount of parallelizable operations during the training of RNNs. To address the first weakness, we propose a new unsupervised network architecture, called AC-SUM-GAN, which formulates the selection of important video fragments as a sequence generation task and learns this task by embedding an Actor-Critic model in a Generative Adversarial Network. The feedback of a trainable Discriminator is used as a reward by the Actor-Critic model in order to explore a space of actions and learn a value function (Critic) and a policy (Actor) for video fragment selection. To tackle the remaining weaknesses, we investigate the use of attention mechanisms for video summarization and propose a new supervised network architecture, called PGL-SUM, that combines global and local multi-head attention mechanisms which take into account the temporal position of the video frames, in order to discover different modelings of the frames' dependencies at different levels of granularity. Based on the acquired experience, we then propose a new unsupervised network architecture, called CA-SUM, which estimates the frames' importance using a novel concentrated attention mechanism that focuses on non-overlapping blocks in the main diagonal of the attention matrix and takes into account the attentive uniqueness and diversity of the associated frames of the video. All the proposed architectures have been extensively evaluated on the most commonly-used benchmark datasets, demonstrating their competitiveness against other approaches and documenting the contribution of our proposals on advancing the current state-of-the-art on video summarization. Finally, we make a first attempt on producing explanations for the video summarization results. Inspired by relevant works in the Natural Language Processing domain, we propose an attention-based method for explainable video summarization and we evaluate the performance of various explanation signals using our CA-SUM architecture and two benchmark datasets for video summarization. The experimental results indicate the advanced performance of explanation signals formed using the inherent attention weights, and demonstrate the ability of the proposed method to explain the video summarization results using clues about the focus of the attention mechanism
Second-Person Surveillance: Politics of User Implication in Digital Documentaries
This dissertation analyzes digital documentaries that utilize second-person address and roleplay to make users feel implicated in contemporary refugee crises, mass incarceration in the U.S., and state and corporate surveillances. Digital documentaries are seemingly more interactive and participatory than linear film and video documentary as they are comprised of a variety of auditory, visual, and written media, utilize networked technologies, and turn the documentary audience into a documentary user. I draw on scholarship from documentary, game, new media, and surveillance studies to analyze how second-person address in digital documentaries is configured through user positioning and direct address within the works themselves, in how organizations and creators frame their productions, and in how users and players respond in reviews, discussion forums, and Letâs Plays. I build on Michael Rothbergâs theorization of the implicated subject to explore how these digital documentaries bring the user into complicated relationality with national and international crises. Visually and experientially implying that users bear responsibility to the subjects and subject matter, these works can, on the one hand, replicate modes of liberal empathy for suffering, distant âothersâ and, on the other, simulate oneâs own surveillant modes of observation or behavior to mirror it back to users and open up oneâs offline thoughts and actions as a site of critique.
This dissertation charts how second-person address shapes and limits the political potentialities of documentary projects and connects them to a lineage of direct address from educational and propaganda films, museum exhibits, and serious games. By centralizing the userâs individual experience, the interventions that second-person digital documentaries can make into social discourse change from public, institution-based education to more privatized forms of sentimental education geared toward personal edification and self-realization. Unless tied to larger initiatives or movements, I argue that digital documentaries reaffirm a neoliberal politics of individual self-regulation and governance instead of public education or collective, social intervention.
Chapter one focuses on 360-degree virtual reality (VR) documentaries that utilize the feeling of presence to position users as if among refugees and as witnesses to refugee experiences in camps outside of Europe and various dwellings in European cities. My analysis of Clouds Over Sidra (Gabo Arora and Chris Milk 2015) and The Displaced (Imraan Ismail and Ben C. Solomon 2015) shows how these VR documentaries utilize observational realism to make believable and immersive their representations of already empathetic refugees. The empathetic refugee is often young, vulnerable, depoliticized and dehistoricized and is a well-known trope in other forms of humanitarian media that continues into VR documentaries. Forced to Flee (Zahra Rasool 2017), I am Rohingya (Zahra Rasool 2017), So Leben FlĂŒchtlinge in Berlin (Berliner Morgenpost 2017), and Limbo: A Virtual Experience of Waiting for Asylum (Shehani Fernando 2017) disrupt easy immersions into realistic-looking VR experiences of stereotyped representations and user identifications and, instead, can reflect back the userâs political inaction and surveillant modes of looking.
Chapter two analyzes web- and social media messenger-based documentaries that position users as outsiders to U.S. mass incarceration. Users are noir-style co-investigators into the crime of the prison-industrial complex in Fremont County, Colorado in Prison Valley: The Prison Industry (David Dufresne and Philippe Brault 2009) and co-riders on a bus transporting prison inmatesâ loved ones for visitations to correctional facilities in Upstate New York in A Temporary Contact (Nirit Peled and Sara Kolster 2017). Both projects construct an experience of carceral constraint for users to reinscribe seeming âoutsideâ places, people, and experiences as within the continuation of the racialized and classed politics of state control through mass incarceration. These projects utilize interfaces that create a tension between replicating an exploitative hierarchy between non-incarcerated users and those subject to mass incarceration while also de-immersing users in these experiences to mirror back the userâs supposed distance from this mode of state regulation.
Chapter three investigates a type of digital game I term dataveillance simulation games, which position users as surveillance agents in ambiguously dystopian nation-states and force users to use their own critical thinking and judgment to construct the criminality of state-sanctioned surveillance targets. Project Perfect Citizen (Bad Cop Studios 2016), Orwell: Keeping an Eye on You (Osmotic Studios 2016), and Papers, Please (Lucas Pope 2013) all create a dual empathy: players empathize with bureaucratic surveillance agents while empathizing with surveillance targets whose emails, text messages, documents, and social media profiles reveal them to be ânormalâ people. I argue that while these games show criminality to be a construct, they also utilize a racialized fear of the loss of oneâs individual privacy to make players feel like they too could be surveillance targets.
Chapter four examines personalized digital documentaries that turn users and their data into the subject matter. Do Not Track (Brett Gaylor 2015), A Week with Wanda (Joe Derry Hall 2019), Stealing Ur Feelings (Noah Levenson 2019), Alfred Premium (JoĂ«l Ronez, Pierre Corbinais, and Ămilie F. Grenier 2019), How They Watch You (Nick Briz 2021), and Fairly Intelligentâą (A.M. Darke 2021) track, monitor, and confront users with their own online behavior to reflect back a corporate surveillance that collects, analyzes, and exploits user data for profit. These digital documentaries utilize emotional fear- and humor-based appeals to persuade users that these technologies are controlling them, shaping their desires and needs, and dehumanizing them through algorithmic surveillance
Blending the Material and Digital World for Hybrid Interfaces
The development of digital technologies in the 21st century is progressing continuously and new device classes such as tablets, smartphones or smartwatches are finding their way into our everyday lives. However, this development also poses problems, as these prevailing touch and gestural interfaces often lack tangibility, take little account of haptic qualities and therefore require full attention from their users. Compared to traditional tools and analog interfaces, the human skills to experience and manipulate material in its natural environment and context remain unexploited. To combine the best of both, a key question is how it is possible to blend the material world and digital world to design and realize novel hybrid interfaces in a meaningful way. Research on Tangible User Interfaces (TUIs) investigates the coupling between physical objects and virtual data. In contrast, hybrid interfaces, which specifically aim to digitally enrich analog artifacts of everyday work, have not yet been sufficiently researched and systematically discussed.
Therefore, this doctoral thesis rethinks how user interfaces can provide useful digital functionality while maintaining their physical properties and familiar patterns of use in the real world. However, the development of such hybrid interfaces raises overarching research questions about the design: Which kind of physical interfaces are worth exploring? What type of digital enhancement will improve existing interfaces? How can hybrid interfaces retain their physical properties while enabling new digital functions? What are suitable methods to explore different design? And how to support technology-enthusiast users in prototyping?
For a systematic investigation, the thesis builds on a design-oriented, exploratory and iterative development process using digital fabrication methods and novel materials. As a main contribution, four specific research projects are presented that apply and discuss different visual and interactive augmentation principles along real-world applications. The applications range from digitally-enhanced paper, interactive cords over visual watch strap extensions to novel prototyping tools for smart garments. While almost all of them integrate visual feedback and haptic input, none of them are built on rigid, rectangular pixel screens or use standard input modalities, as they all aim to reveal new design approaches. The dissertation shows how valuable it can be to rethink familiar, analog applications while thoughtfully extending them digitally. Finally, this thesisâ extensive work of engineering versatile research platforms is accompanied by overarching conceptual work, user evaluations and technical experiments, as well as literature reviews.Die Durchdringung digitaler Technologien im 21. Jahrhundert schreitet stetig voran und neue GerĂ€teklassen wie Tablets, Smartphones oder Smartwatches erobern unseren Alltag. Diese Entwicklung birgt aber auch Probleme, denn die vorherrschenden berĂŒhrungsempfindlichen OberflĂ€chen berĂŒcksichtigen kaum haptische QualitĂ€ten und erfordern daher die volle Aufmerksamkeit ihrer Nutzer:innen. Im Vergleich zu traditionellen Werkzeugen und analogen Schnittstellen bleiben die menschlichen FĂ€higkeiten ungenutzt, die Umwelt mit allen Sinnen zu begreifen und wahrzunehmen. Um das Beste aus beiden Welten zu vereinen, stellt sich daher die Frage, wie neuartige hybride Schnittstellen sinnvoll gestaltet und realisiert werden können, um die materielle und die digitale Welt zu verschmelzen. In der Forschung zu Tangible User Interfaces (TUIs) wird die Verbindung zwischen physischen Objekten und virtuellen Daten untersucht. Noch nicht ausreichend erforscht wurden hingegen hybride Schnittstellen, die speziell darauf abzielen, physische GegenstĂ€nde des Alltags digital zu erweitern und anhand geeigneter Designparameter und EntwurfsrĂ€ume systematisch zu untersuchen.
In dieser Dissertation wird daher untersucht, wie MaterialitĂ€t und DigitalitĂ€t nahtlos ineinander ĂŒbergehen können. Es soll erforscht werden, wie kĂŒnftige Benutzungsschnittstellen nĂŒtzliche digitale Funktionen bereitstellen können, ohne ihre physischen Eigenschaften und vertrauten Nutzungsmuster in der realen Welt zu verlieren. Die Entwicklung solcher hybriden AnsĂ€tze wirft jedoch ĂŒbergreifende Forschungsfragen zum Design auf: Welche Arten von physischen Schnittstellen sind es wert, betrachtet zu werden? Welche Art von digitaler Erweiterung verbessert das Bestehende? Wie können hybride Konzepte ihre physischen Eigenschaften beibehalten und gleichzeitig neue digitale Funktionen ermöglichen? Was sind geeignete Methoden, um verschiedene Designs zu erforschen? Wie kann man Technologiebegeisterte bei der Erstellung von Prototypen unterstĂŒtzen?
FĂŒr eine systematische Untersuchung stĂŒtzt sich die Arbeit auf einen designorientierten, explorativen und iterativen Entwicklungsprozess unter Verwendung digitaler Fabrikationsmethoden und neuartiger Materialien. Im Hauptteil werden vier Forschungsprojekte vorgestellt, die verschiedene visuelle und interaktive Prinzipien entlang realer Anwendungen diskutieren. Die Szenarien reichen von digital angereichertem Papier, interaktiven Kordeln ĂŒber visuelle Erweiterungen von UhrarmbĂ€ndern bis hin zu neuartigen Prototyping-Tools fĂŒr intelligente KleidungsstĂŒcke. Um neue DesignansĂ€tze aufzuzeigen, integrieren nahezu alle visuelles Feedback und haptische Eingaben, um Alternativen zu Standard-EingabemodalitĂ€ten auf starren Pixelbildschirmen zu schaffen. Die Dissertation hat gezeigt, wie wertvoll es sein kann, bekannte, analoge Anwendungen zu ĂŒberdenken und sie dabei gleichzeitig mit Bedacht digital zu erweitern. Dabei umfasst die vorliegende Arbeit sowohl realisierte technische Forschungsplattformen als auch ĂŒbergreifende konzeptionelle Arbeiten, Nutzerstudien und technische Experimente sowie die Analyse existierender Forschungsarbeiten
Informationsströme in digitalen Kulturen
Wir sind umgeben von einer Vielzahl an Informationsströmen, die uns selbstverstĂ€ndlich erscheinen. Um diese digitalen Kulturen zu beschreiben, entwickeln medienwissenschaftliche Arbeiten Theorien einer Welt im Fluss. Dabei erliegen ihre Diagnosen oftmals einem Technikfetisch und vernachlĂ€ssigen gesellschaftliche Strukturen. Mathias Denecke legt eine systematische Kritik dieser Theoriebildung vor. Dazu zeichnet er die Geschichte der Rede von strömenden Informationen in der Entwicklung digitaler Computer nach und diskutiert, wie der Begriff fĂŒr Gegenwartsbeschreibungen produktiv gemacht werden kann
Handbuch Zeitschriftenforschung
Wie werden Gesellschaften in Zeitschriften betrachtet und welche Ăbersetzungsleistungen bieten jene in unterschiedlichen gesellschaftlichen Kontexten? Um diesen Fragen nachzugehen, analysieren die BeitrĂ€ger*innen des Bandes das Zusammenspiel von Textgestaltung, Design, Inhalten, Infrastrukturen und Zielgruppen von Zeitschriften. Sie erweitern den Blick im Rahmen einer differenzierungstheoretischen Forschungsagenda und betrachten Zeitschriften als materialisierte Zeichensysteme und kommunikative Artefakte innerhalb der materiellen Kultur der Gesellschaft
Design of an E-learning system using semantic information and cloud computing technologies
Humanity is currently suffering from many difficult problems that threaten the life and survival of the human race. It is very easy for all mankind to be affected, directly or indirectly, by these problems. Education is a key solution for most of them. In our thesis we tried to make use of current technologies to enhance and ease the learning process.
We have designed an e-learning system based on semantic information and cloud computing, in addition to many other technologies that contribute to improving the educational process and raising the level of students. The design was built after much research on useful technology, its types, and examples of actual systems that were previously discussed by other researchers.
In addition to the proposed design, an algorithm was implemented to identify topics found in large textual educational resources. It was tested and proved to be efficient against other methods. The algorithm has the ability of extracting the main topics from textual learning resources, linking related resources and generating interactive dynamic knowledge graphs. This algorithm accurately and efficiently accomplishes those tasks even for bigger books. We used Wikipedia Miner, TextRank, and Gensim within our algorithm. Our algorithmâs accuracy was evaluated against Gensim, largely improving its accuracy.
Augmenting the system design with the implemented algorithm will produce many useful services for improving the learning process such as: identifying main topics of big textual learning resources automatically and connecting them to other well defined concepts from Wikipedia, enriching current learning resources with semantic information from external sources, providing student with browsable dynamic interactive knowledge graphs, and making use of learning groups to encourage students to share their learning experiences and feedback with other learners.Programa de Doctorado en IngenierĂa TelemĂĄtica por la Universidad Carlos III de MadridPresidente: Luis SĂĄnchez FernĂĄndez.- Secretario: Luis de la Fuente ValentĂn.- Vocal: Norberto FernĂĄndez GarcĂ
Empowering Information Systems Users: The Role of Timely and Customizable Information for User Engagement and Selection Behavior
Information systems (IS) increasingly empower their users by strengthening usersâ capability and autonomy to make their own decisions how to use and engage with IS. Specifically, users are empowered when they have sufficient knowledge to make rational decisions within IS and sufficient control to shape their experience with IS. In line with these pillars of empowerment, technological advancements unlock new possibilities for IS providers to empower users with access to high quality information (e.g., by providing timely updates of dynamically changing information) and with the ability to control the information stream (e.g., by implementing interfaces to customize websites). As a result, users have greater autonomy to actively shape their user experience to their likening, making them less dependent on having to identify IS that match their needs. At the same time, empowering users pays off for IS providers, as empowered users are known to form more positive attitudes and intentions to engage with the empowering IS. This thesis addresses the two aforementioned pillars of empowerment through knowledge and empowerment through control. Four studies shed light on how the increasingly prevalent practice of empowering users with timely and customizable information affects user engagement as well as usersâ selection behavior.
The first strand of this thesis investigates user empowerment through timely information in the context of decision support systems (DSS) that aid users in their selection of which (physical) location to visit. To avoid congestion at locations, such DSS communicate how busy each location is by displaying crowding information (CI), accompanied by timeliness cues indicating when this CI was retrieved (e.g., âupdated just nowâ vs. âaverage over the last yearâ). Helping users avoid crowded locations becomes all the more important during periods of extraordinary pathogenic risk, such as the COVID-19 pandemic, where physical distancing is imperative for the containment of the pathogen. Against this background, the first study in this thesis investigates how CI with different levels of timeliness affects how users select between differently crowded medical practices. The results demonstrate that while the display of CI is generally useful for users to avoid crowded locations, providing particularly timely CI (i.e., updated close to real-time) leads users to select less crowded locations even more effectively. Moreover, this effect is strongest for individuals who exhibit low levels of health anxiety â an important contextual variable influencing user behavior during the COVID-19 pandemic.
The second study extends the findings of the first study by investigating a context in which hedonic motives may encourage users to seek instead of avoid crowds. Specifically, the study examines how timely CI affects usersâ choice between differently crowded bars. Despite users longing for the presence of others as part of their visit experience, the results show that particularly timely CI makes users more aware of potential costs of congestion (e.g., prolonged wait times) and consequently leads users to select less crowded locations â thereby corroborating the previous findings in the utilitarian context of selecting a medical practice. Importantly, timelier CI also increases user engagement in that users express a greater intention to reuse the DSS providing the CI. This finding indicates that timely CI not only contributes to the containment of congestion, but also allows DSS providers to retain users more effectively and thereby achieve recurring impact on the reduction of crowding.
The second strand of this thesis investigates user empowerment through customizable information in the context of (banner) ads on websites. As ads oftentimes cause irritation and stifle user engagement with the website, first website providers have begun to empower users to customize how many ads they agree to have displayed. Despite website providers hoping to thereby enhance user engagement, it is unclear how users respond to the ability to customize ads they never asked for. Against this backdrop, the third study investigates how the provision of ad quantity customization (AQC) affects user engagement and which ad quantity levels users opt for. The results demonstrate that offering AQC consistently enhances user engagement in that users with access to AQC stay longer on the website and visit more sub-pages than users who cannot customize ad quantity. Counter-intuitively, a website with ads that offers AQC elicits even greater user engagement than a website that is entirely free of ads by default. In addition, the effect on user engagement is strongest for users accessing the website with a mobile (vs. stationary) device. Interestingly, users do not configure AQC to eliminate ads altogether, but instead opt for 29.0% of the default amount of ads to be displayed.
The fourth study seeks to extend the previous findings by shedding light on the underlying mechanism that drives the effect of providing AQC on user engagement. The findings suggest that offering AQC elicits perceived empowerment as a pivotal stimulant with two important outcomes: First, users pay closer attention to the website, thereby discovering more information useful to them and consequently experiencing a greater fit between the websiteâs information and their own needs. Second, the feeling of being in control over ads, as typically immutable and irritating website elements, elicits a sense of enjoyment. Both informational fit and perceived enjoyment then lead users to engage more intensely with the website.
Overall, this thesis showcases the role and importance of IS-enabled user empowerment by providing a more comprehensive understanding of how empowering users with timely and customizable information affects user engagement and usersâ selection behavior. In doing so, this thesis answers calls for research that urge scholars to not only shed light on emerging phenomena, but also to enable and empower IS users. The studies in this thesis contribute to IS research on empowerment by (1) revealing the importance of timeliness of information as a thus far under-investigated source of empowerment and by (2) uncovering ad customization as a hitherto largely neglected, yet important piece of web customization that complements our understanding of empowerment mechanisms. In addition, this thesis also offers valuable insights and actionable recommendations how DSS providers and policy makers can harness empowerment through timely CI to recurringly reduce crowding without infringing on usersâ freedom. Likewise, this thesis guides website providers how to leverage ads as website elements that users enjoy to customize to boost user engagement with the website as a whole
WearPut : Designing Dexterous Wearable Input based on the Characteristics of Human Finger Motions
Department of Biomedical Engineering (Human Factors Engineering)Powerful microchips for computing and networking allow a wide range of wearable devices to be miniaturized with high fidelity and availability. In particular, the commercially successful smartwatches placed on the wrist drive market growth by sharing the role of smartphones and health management. The emerging Head Mounted Displays (HMDs) for Augmented Reality (AR) and Virtual Reality (VR) also impact various application areas in video games, education, simulation, and productivity tools. However, these powerful wearables have challenges in interaction with the inevitably limited space for input and output due to the specialized form factors for fitting the body parts. To complement the constrained interaction experience, many wearable devices still rely on other large form factor devices (e.g., smartphones or hand-held controllers). Despite their usefulness, the additional devices for interaction can constrain the viability of wearable devices in many usage scenarios by tethering users' hands to the physical devices. This thesis argues that developing novel Human-Computer interaction techniques for the specialized wearable form factors is vital for wearables to be reliable standalone products.
This thesis seeks to address the issue of constrained interaction experience with novel interaction techniques by exploring finger motions during input for the specialized form factors of wearable devices. The several characteristics of the finger input motions are promising to enable increases in the expressiveness of input on the physically limited input space of wearable devices. First, the input techniques with fingers are prevalent on many large form factor devices (e.g., touchscreen or physical keyboard) due to fast and accurate performance and high familiarity. Second, many commercial wearable products provide built-in sensors (e.g., touchscreen or hand tracking system) to detect finger motions. This enables the implementation of novel interaction systems without any additional sensors or devices. Third, the specialized form factors of wearable devices can create unique input contexts while the fingers approach their locations, shapes, and components. Finally, the dexterity of fingers with a distinctive appearance, high degrees of freedom, and high sensitivity of joint angle perception have the potential to widen the range of input available with various movement features on the surface and in the air. Accordingly, the general claim of this thesis is that understanding how users move their fingers during input will enable increases in the expressiveness of the interaction techniques we can create for resource-limited wearable devices.
This thesis demonstrates the general claim by providing evidence in various wearable scenarios with smartwatches and HMDs. First, this thesis explored the comfort range of static and dynamic touch input with angles on the touchscreen of smartwatches. The results showed the specific comfort ranges on variations in fingers, finger regions, and poses due to the unique input context that the touching hand approaches a small and fixed touchscreen with a limited range of angles. Then, finger region-aware systems that recognize the flat and side of the finger were constructed based on the contact areas on the touchscreen to enhance the expressiveness of angle-based touch input. In the second scenario, this thesis revealed distinctive touch profiles of different fingers caused by the unique input context for the touchscreen of smartwatches. The results led to the implementation of finger identification systems for distinguishing two or three fingers. Two virtual keyboards with 12 and 16 keys showed the feasibility of touch-based finger identification that enables increases in the expressiveness of touch input techniques. In addition, this thesis supports the general claim with a range of wearable scenarios by exploring the finger input motions in the air. In the third scenario, this thesis investigated the motions of in-air finger stroking during unconstrained in-air typing for HMDs. The results of the observation study revealed details of in-air finger motions during fast sequential input, such as strategies, kinematics, correlated movements, inter-fingerstroke relationship, and individual in-air keys. The in-depth analysis led to a practical guideline for developing robust in-air typing systems with finger stroking. Lastly, this thesis examined the viable locations of in-air thumb touch input to the virtual targets above the palm. It was confirmed that fast and accurate sequential thumb touch can be achieved at a total of 8 key locations with the built-in hand tracking system in a commercial HMD. Final typing studies with a novel in-air thumb typing system verified increases in the expressiveness of virtual target selection on HMDs.
This thesis argues that the objective and subjective results and novel interaction techniques in various wearable scenarios support the general claim that understanding how users move their fingers during input will enable increases in the expressiveness of the interaction techniques we can create for resource-limited wearable devices. Finally, this thesis concludes with thesis contributions, design considerations, and the scope of future research works, for future researchers and developers to implement robust finger-based interaction systems on various types of wearable devices.ope
- âŠ