Emerging ExG-based NUI Inputs in Extended Realities : A Bottom-up Survey

Incremental and quantitative improvements of two-way interactions with extended realities (XR) are contributing toward a qualitative leap into a state of XR ecosystems being efficient, user-friendly, and widely adopted. However, there are multiple barriers on the way toward the omnipresence of XR; among them are the following: computational and power limitations of portable hardware, social acceptance of novel interaction protocols, and usability and efficiency of interfaces. In this article, we overview and analyse novel natural user interfaces based on sensing electrical bio-signals that can be leveraged to tackle the challenges of XR input interactions. Electroencephalography-based brain-machine interfaces that enable thought-only hands-free interaction, myoelectric input methods that track body gestures employing electromyography, and gaze-tracking electrooculography input interfaces are the examples of electrical bio-signal sensing technologies united under a collective concept of ExG. ExG signal acquisition modalities provide a way to interact with computing systems using natural intuitive actions enriching interactions with XR. This survey will provide a bottom-up overview starting from (i) underlying biological aspects and signal acquisition techniques, (ii) ExG hardware solutions, (iii) ExG-enabled applications, (iv) discussion on social acceptance of such applications and technologies, as well as (v) research challenges, application directions, and open problems; evidencing the benefits that ExG-based Natural User Interfaces inputs can introduceto the areaof XR.Peer reviewe

Emerging ExG-based NUI Inputs in Extended Realities : A Bottom-up Survey

Incremental and quantitative improvements of two-way interactions with extended realities (XR) are contributing toward a qualitative leap into a state of XR ecosystems being efficient, user-friendly, and widely adopted. However, there are multiple barriers on the way toward the omnipresence of XR; among them are the following: computational and power limitations of portable hardware, social acceptance of novel interaction protocols, and usability and efficiency of interfaces. In this article, we overview and analyse novel natural user interfaces based on sensing electrical bio-signals that can be leveraged to tackle the challenges of XR input interactions. Electroencephalography-based brain-machine interfaces that enable thought-only hands-free interaction, myoelectric input methods that track body gestures employing electromyography, and gaze-tracking electrooculography input interfaces are the examples of electrical bio-signal sensing technologies united under a collective concept of ExG. ExG signal acquisition modalities provide a way to interact with computing systems using natural intuitive actions enriching interactions with XR. This survey will provide a bottom-up overview starting from (i) underlying biological aspects and signal acquisition techniques, (ii) ExG hardware solutions, (iii) ExG-enabled applications, (iv) discussion on social acceptance of such applications and technologies, as well as (v) research challenges, application directions, and open problems; evidencing the benefits that ExG-based Natural User Interfaces inputs can introduceto the areaof XR.Peer reviewe

Interagir en l'absence de signifiants : le cas des swhidgets

At the heart of this thesis is a common but problematic situation that users of digital systems often face in their daily interactions: to interact with the system, they need some knowledge of an interaction possibility, some piece of information about the interface, but this information is not provided in the context in which they need it. I call such interaction possibilities non-signified, and signifier-less designs the interfaces and interaction techniques that rely on non-signified interaction possibilities.An example of modern signifier-less design is what I call "swhidgets" for "SWIpe-revealed HIDden WIDGETS": widgets that are hidden under the screen bezels or other interface elements, out of view and not advertised by any graphical mark, but that can be revealed by dragging them into view with a swipe gesture relying on a physical manipulation metaphor. Swhidgets are an important component of touch-based smartphone and tablets interfaces, and will be the principal signifier-less design studied in this thesis.When facing a signifier-less design, users may be confused about what they should do and how to achieve their goals; or they might have to use suboptimal ways of achieving their goals because they are unaware of the existence of more efficient options. It is thus usually advised to avoid signifier-less designs. Yet, despite designers’ awareness of the problems they may cause, signifier-less designs are common in user interfaces. They thus deserve a deeper analysis than simply advising to avoid them in interface design. Indeed, there might be good reasons to apply this design: maybe they provide some benefits that are hard to see with our current understanding of these designs, or maybe there is no way to avoid them.In this thesis, I study the question of why designers would create interfaces that do not clearly expose some of their interaction possibilities, taking the case of swhidgets as an example and focus of inquiry. As a preliminary work on swhidgets, I focus on the following questions: What are signifier-less designs and what aspects of swhidgets design make them unique? Do users know the swhidgets provided by their system? How did they get to know them despite their lack of signifiers? What are the benefits of not having signifiers in the design of swhidgets?My contributions to these questions are:- I define signifier-less designs and provide observations of this type of design in user interfaces.- I provide an analysis of the fundamental notions required to define signifier-less designs: affordances, signifiers and semiotics.- I propose a model of user discovery and adoption of interaction techniques in general, relying on three dimensions and their relationships: users’ current knowledge and skills, users’ motivations, and the design means of informing users provided by the interfaces.- I propose the notions of Degree of Knowledge and Source of Knowledge derived from this model, that can be used in experiments to evaluate how well the participants know an interaction technique and how they discovered it.- I present the design and results of two studies on iOS swhidgets that investigate how well users known them, how they discover them, their reasons for not using them, how they generally feel about them, and how they integrate them in the way they think about their interactions with the system. These studies revealed that swhidgets were globally appreciated and relatively well known by users, although there is still room for improvement, notably for some specific swhidgets.I conclude with perspective for future works regarding the transfer of knowledge about swhidgets from one application to another, the pertinence of considering all aspects of user experience to understand the design of swhidgets, and the possibility to increase the discoverability of swhidgets by using animated transitions between interface views.Le cas d'un utilisateur confronté à une interface qui ne l'informe pas d'une possibilité d'interaction au moment où il en a besoin est un problème fondamental d'IHM. La présence de telles possibilités d'interaction em non-signalées est fréquente dans les interfaces homme-machine modernes et potentiellement problématique, rendant nécessaire l'étude des interfaces et techniques d'interaction dites sans signifiants.Un exemple de conception "sans signifiants" moderne est ce que j'appelle un Swhidget pour "SWIpe-revealed HIDden WIDGET": un composant d'interface normalement caché sous les bords de l'écran ou sous un autre objet, pouvant être révélé en le tirant à l'aide d'un geste de balayage selon une métaphore de manipulation physique. Les Swhidgets sont des composants importants des interfaces de téléphones et tablettes à écran tactile, et sont la principale conception sans signifiant étudiée dans cette thèse.En présence d'une conception sans signifiant, les utilisateurs peuvent être confus quant à ce qu'ils doivent faire pour atteindre leur but, ou être réduits à utiliser des méthodes sous-optimales parce qu'ils ne sont pas conscients de l'existence de meilleures alternatives. Il est donc généralement recommandé d'éviter de concevoir des interfaces sans signifiants. De telles interfaces sont pourtant courantes bien que les concepteurs soient conscient des problèmes qu'elles causent. Elles méritent donc une analyse plus approfondie, au delà du simple conseil de les éviter. En effet, il pourrait y avoir de bonnes raisons de concevoir des interfaces sans signifiants, qu'elles aient des qualités difficiles à mettre en évidence en l'état actuel de notre compréhension ou qu'il soit simplement impossible de les éviter.Dans cette thèse, j'analyse les raisons pouvant inciter à la conception d'interfaces qui n'exposent pas clairement les possibilités d'interaction qu'elles offrent, en prenant les Swhidgets comme objet d'étude principal. Pour cette étude initiale des Swhidgets, je me concentre sur les points suivants : que sont les conceptions sans signifiants et quels aspects des Swhidgets leur sont propres ? Les utilisateurs connaissent-ils les Swhidgets de leurs systèmes ? Comment les ont-ils connus malgré l'absence de signifiants ? Quels avantages y a-t-il à ne pas avoir de signifiants ?Les contributions de cette thèse sur ces points sont :- Une definition des conceptions sans signifiants basée sur des observations de telles conceptions dans des interfaces. - Une analyse des notions fondamentales requises pour la définition des conceptions sans signifiants: affordances, signifiants et sémiotique.- Un modèle de la découverte et adoption par les utilisateurs de techniques d'interaction en général, reposant sur trois dimensions et leurs relations: les compétences et connaissances actuelles des utilisateurs, leur motivations, et les moyens par lesquels une interface peut informer ses utilisateurs.- Les notions de degré de connaissance et de source de connaissance, dérivées de ce modèle, qui permettent d'évaluer expérimentalement à quel point les utilisateurs connaissent une technique d'interaction et comment ils l'ont découverte.- La conception et les résultats de deux expériences sur les Swhidgets d'iOS pour évaluer la connaissance qu'en ont les utilisateurs, comment ils les ont découvert, leurs éventuelles raisons de ne pas les utiliser, comment ils les perçoivent globalement et les intègrent dans leur façon de penser l'interaction. Ces études montrent que les Swhidgets sont globalement appréciés et relativement bien connus, tout en laissant de la place pour des améliorations, surtout pour certains Swhidgets.Cette thèse ouvre des perspectives concernant le transfert de connaissances entre applications, la pertinence du concept d'expérience utilisateur pour la compréhension des Swhidgets, et la possibilité de favoriser leur découverte lors de transitions animées entre vues