Review of three-dimensional human-computer interaction with focus on the leap motion controller

Modern hardware and software development has led to an evolution of user interfaces from command-line to natural user interfaces for virtual immersive environments. Gestures imitating real-world interaction tasks increasingly replace classical two-dimensional interfaces based on Windows/Icons/Menus/Pointers (WIMP) or touch metaphors. Thus, the purpose of this paper is to survey the state-of-the-art Human-Computer Interaction (HCI) techniques with a focus on the special field of three-dimensional interaction. This includes an overview of currently available interaction devices, their applications of usage and underlying methods for gesture design and recognition. Focus is on interfaces based on the Leap Motion Controller (LMC) and corresponding methods of gesture design and recognition. Further, a review of evaluation methods for the proposed natural user interfaces is given

A New Hand-Movement-Based Authentication Method Using Feature Importance Selection with the Hotelling’s Statistic

The growing amount of collected and processed data means that there is a need to control access to these resources. Very often, this type of control is carried out on the basis of biometric analysis. The article proposes a new user authentication method based on a spatial analysis of the movement of the finger’s position. This movement creates a sequence of data that is registered by a motion recording device. The presented approach combines spatial analysis of the position of all fingers at the time. The proposed method is able to use the specific, often different movements of fingers of each user. The experimental results confirm the effectiveness of the method in biometric applications. In this paper, we also introduce an effective method of feature selection, based on the Hotelling T2 statistic. This approach allows selecting the best distinctive features of each object from a set of all objects in the database. It is possible thanks to the appropriate preparation of the input data

Authenticating Users with 3D Passwords Captured by Motion Sensors

Authentication plays a key role in securing various resources including corporate facilities or electronic assets. As the most used authentication scheme, knowledgebased authentication is easy to use but its security is bounded by how much a user can remember. Biometrics-based authentication requires no memorization but ‘resetting’ a biometric password may not always be possible. Thus, we propose study several behavioral biometrics (i.e., mid-air gestures) for authentication which does not have the same privacy or availability concerns as of physiological biometrics. In this dissertation, we first propose a user-friendly authentication system Kin- Write that allows users to choose arbitrary, short and easy-to-memorize passwords while providing resilience to password cracking and password theft. Specifically, we let users write their passwords (i.e., signatures in the 3D space), and verify a user’s identity with similarities between the user’s password and enrolled password templates. Dynamic time warping distance is used for similarity calculation between 3D passwords samples. In the second part of the dissertation, we design an authentication scheme that does not depend on the handwriting contents, i.e., regardless of the written words or symbols, and adapt challenge-response mechanism to avoid possible eavesdropping, man-in-the-middle attacks, and reply attacks. We design a MoCRA system that utilizes Leap Motion to capture users’ writing movements and use writing style to verify users, even if what they write during the verification is completely different from what they write during the enrollment. Specifically, MoCRA leverages co-occurrence matrices to model the handwriting styles, and use a Support Vector Machine (SVM) to accept a legitimate user and reject the rest. In the third part, we study both security and usability performance on multiple types of mid-air gestures that used as passwords, including writing signatures in the air. We objectively quantify the usability performance by metrics related to the enroll time and the complexity of the gestures, and evaluate the security performance by the authentication performance. In addition, we subjectively evaluate the gestures by survey responses from both field subjects who participated in gesture experiments and on-line subjects who watched a short video on gesture introducing. Finally, we study the consistency of gestures over samples collected in a two-month period, and evaluate their security under shoulder surfing attacks

Contactless User Authentification Using Leap Motion Sensor

Bezkontaktná autentizácia užívateľov naberá ako nová technológia na popularite. Poznatky v spracovaní obrazu a ich aplikácia prispeli k vzniku zariadení akým je ovládač Leap Motion. Toto zariadenie od firmy s rovnakým názvom Leap Motion inc. dokáže určovať polohu ruky v priestore a rozpoznávať jednoduché gestá. Táto práca sa zaoberá aplikáciou dát z rozhrania tohto ovládača pri identifikácii osôb. Využité sú už overené charakteristiky geometrie ruky, ako šírky a dĺžky prstov. Cieľ práce je najmä overiť použitie tohto na trhu dostupného a cenovo nenáročného senzoru pri rozpoznávaní. Užívateľ si toto zariadenie pravdepodobne nezakúpi kvôli jeho vlastnosti autentifikácie osôb, no môže využívať túto jednoduchú formu zabezpečenia bez akejkoľvek ďalšej investície. Nakoniec, výsledky práce ukazujú presnosť okolo 99 % na menšej vzorke ľudí, ktorá pripomína práve domáce použitie.Contactless authentication of users has grown in popularity as a new technology. Recent findings in the field of computer vision and its applications have contributed to the emergence of new devices such as Leap Motion controller. This device is capable of precise recognition of hand positions and simple gestures identification. This paper presents an application of data gathered from controller's interface and using them for user identification. Proposed solution uses hand geometry to evaluate the recognition process where this accessible and inexpensive device can be used. Therefore, the user can easily benefit from this extra feature coming with the new device. To conclude, the results show overall accuracy over 99 % on a relatively small dataset.

User-based gesture vocabulary for form creation during a product design process

There are inconsistencies between the nature of the conceptual design and the functionalities of the computational systems supporting it, which disrupt the designers’ process, focusing on technology rather than designers’ needs. A need for elicitation of hand gestures appropriate for the requirements of the conceptual design, rather than those arbitrarily chosen or focusing on ease of implementation was identified.The aim of this thesis is to identify natural and intuitive hand gestures for conceptual design, performed by designers (3rd, 4th year product design engineering students and recent graduates) working on their own, without instruction and without limitations imposed by the facilitating technology. This was done via a user centred study including 44 participants. 1785 gestures were collected. Gestures were explored as a sole mean for shape creation and manipulation in virtual 3D space. Gestures were identified, described in writing, sketched, coded based on the taxonomy used, categorised based on hand form and the path travelled and variants identified. Then they were statistically analysed to ascertain agreement rates between the participants, significance of the agreement and the likelihood of number of repetitions for each category occurring by chance. The most frequently used and statistically significant gestures formed the consensus set of vocabulary for conceptual design. The effect of the shape of the manipulated object on the gesture performed, and if the sequence of the gestures participants proposed was different from the established CAD solid modelling practices were also observed.Vocabulary was evaluated by non-designer participants, and the outcomes have shown that the majority of gestures were appropriate and easy to perform. Evaluation was performed theoretically and in the VR environment. Participants selected their preferred gestures for each activity, and a variant of the vocabulary for conceptual design was created as an outcome, that aims to ensure that extensive training is not required, extending the ability to design beyond trained designers only.There are inconsistencies between the nature of the conceptual design and the functionalities of the computational systems supporting it, which disrupt the designers’ process, focusing on technology rather than designers’ needs. A need for elicitation of hand gestures appropriate for the requirements of the conceptual design, rather than those arbitrarily chosen or focusing on ease of implementation was identified.The aim of this thesis is to identify natural and intuitive hand gestures for conceptual design, performed by designers (3rd, 4th year product design engineering students and recent graduates) working on their own, without instruction and without limitations imposed by the facilitating technology. This was done via a user centred study including 44 participants. 1785 gestures were collected. Gestures were explored as a sole mean for shape creation and manipulation in virtual 3D space. Gestures were identified, described in writing, sketched, coded based on the taxonomy used, categorised based on hand form and the path travelled and variants identified. Then they were statistically analysed to ascertain agreement rates between the participants, significance of the agreement and the likelihood of number of repetitions for each category occurring by chance. The most frequently used and statistically significant gestures formed the consensus set of vocabulary for conceptual design. The effect of the shape of the manipulated object on the gesture performed, and if the sequence of the gestures participants proposed was different from the established CAD solid modelling practices were also observed.Vocabulary was evaluated by non-designer participants, and the outcomes have shown that the majority of gestures were appropriate and easy to perform. Evaluation was performed theoretically and in the VR environment. Participants selected their preferred gestures for each activity, and a variant of the vocabulary for conceptual design was created as an outcome, that aims to ensure that extensive training is not required, extending the ability to design beyond trained designers only

Watching Androids Dream of Electric Sheep: Immersive Technology, Biometric Psychography, and the Law

Virtual reality and augmented reality present exceedingly complex privacy issues because of the enhanced user experience and reality-based models. Unlike the issues presented by traditional gaming and social media, immersive technology poses inherent risks, which our legal understanding of biometrics and online harassment is simply not prepared to address. This Article offers five important contributions to this emerging space. It begins by introducing a new area of legal and policy inquiry raised by immersive technology called “biometric psychography.” Second, it explains how immersive technology works to a legal audience and defines concepts that are essential to understanding the risks that the technology poses. Third, it analyzes the gaps in privacy law to address biometric psychography and other emerging challenges raised by immersive technology that most regulators and consumers incorrectly assume will be governed by existing law. Fourth, this Article sources firsthand interviews from early innovators and leading thinkers to highlight harassment and user experience risks posed by immersive technology. Finally, this Article compiles insights from each of these discussions to propose a framework that integrates privacy and human rights into the development of future immersive tech applications. It applies that framework to three specific scenarios and demonstrates how it can help navigate challenges, both old and new