PILOT: Password and PIN Information Leakage from Obfuscated Typing Videos

This paper studies leakage of user passwords and PINs based on observations of typing feedback on screens or from projectors in the form of masked characters that indicate keystrokes. To this end, we developed an attack called Password and Pin Information Leakage from Obfuscated Typing Videos (PILOT). Our attack extracts inter-keystroke timing information from videos of password masking characters displayed when users type their password on a computer, or their PIN at an ATM. We conducted several experiments in various attack scenarios. Results indicate that, while in some cases leakage is minor, it is quite substantial in others. By leveraging inter-keystroke timings, PILOT recovers 8-character alphanumeric passwords in as little as 19 attempts. When guessing PINs, PILOT significantly improved on both random guessing and the attack strategy adopted in our prior work [4]. In particular, we were able to guess about 3% of the PINs within 10 attempts. This corresponds to a 26-fold improvement compared to random guessing. Our results strongly indicate that secure password masking GUIs must consider the information leakage identified in this paper

The Data Egg: A new solution to text entry barriers

A unit that allows text entry with only one hand has been developed, and holds the promise of allowing computers to be truly portable. It is unique in that it allows operation in any position, freeing the user from the traditional constraints of having to be seated near a desk. This hand held, chord-key-based unit can be used either autonomously for idea capturing, or tethered to a personal computer and used as an auxiliary keyboard. Astronauts, journalists, the bedridden, and anyone else normally barred from using a computer while on the job could also benefit from this form of man-machine interface, which has been dubbed the 'Data Egg'

Using Hover to Compromise the Confidentiality of User Input on Android

We show that the new hover (floating touch) technology, available in a number of today's smartphone models, can be abused by any Android application running with a common SYSTEM_ALERT_WINDOW permission to record all touchscreen input into other applications. Leveraging this attack, a malicious application running on the system is therefore able to profile user's behavior, capture sensitive input such as passwords and PINs as well as record all user's social interactions. To evaluate our attack we implemented Hoover, a proof-of-concept malicious application that runs in the system background and records all input to foreground applications. We evaluated Hoover with 40 users, across two different Android devices and two input methods, stylus and finger. In the case of touchscreen input by finger, Hoover estimated the positions of users' clicks within an error of 100 pixels and keyboard input with an accuracy of 79%. Hoover captured users' input by stylus even more accurately, estimating users' clicks within 2 pixels and keyboard input with an accuracy of 98%. We discuss ways of mitigating this attack and show that this cannot be done by simply restricting access to permissions or imposing additional cognitive load on the users since this would significantly constrain the intended use of the hover technology.Comment: 11 page

INVESTIGATING MIDAIR VIRTUAL KEYBOARD INPUT USING A HEAD MOUNTED DISPLAY

Until recently text entry in virtual reality has been limited to using hand-held controllers. These techniques of text entry are feasible only for entering short texts like usernames and passwords. But recent improvements in virtual reality devices have paved the way to varied interactions in virtual environment and many of these tasks include annotation, text messaging, etc. These tasks require an effective way of text entry in virtual reality. We present an interactive midair text entry system in virtual reality which allows users to use their one or both hands as the means of entering text. Our system also allows users to enter text on a split keyboard using their two hands. We investigated user performance on these three conditions and found that users were slightly faster when they were using both hands. In this case, the mean entry rate was 16.4 words-per-minute (wpm). While using one hand, the entry rate was 16.1 wpm and using the split keyboard the entry rate was 14.7 wpm. The character error rates (CER) in these conditions were 0.74%, 0.79% and 1.41% respectively. We also examined the extent to which a user can enter text without having any visual feedback of a keyboard i.e. on an invisible keyboard in the virtual environment. While some found it difficult, results were promising for a subset of 15 participants of the 22 participants. The subset had a mean entry rate of 10.0 wpm and a mean error rate of 2.98%

TWO-HANDED TYPING METHOD ON AN ARBITRARY SURFACE

A computing device may detect user input, such as finger movements resembling typing on an invisible virtual keyboard in the air or on any surface, to enable typing. The computing device may use sensors (e.g., accelerometers, cameras, piezoelectric sensors, etc.) to detect the user’s finger movements, such as the user’s fingers moving through the air and/or contacting a surface. The computing device may then decode (or, in other words, convert, interpret, analyze, etc.) the detected finger movements to identify corresponding inputs representative of characters (e.g., alphanumeric characters, national characters, special characters, etc.). To reduce input errors, the computing device may decode the detected finger movements, at least in part, based on contextual information, such as preceding characters, words, and/or the like entered via previously detected user inputs. Similarly, the computing device may apply machine learning techniques and adjust parameters, such as a signal-to-noise ratio, to improve the accuracy of input-entry. In some examples, the computing device may implement specific recognition, prediction, and correction algorithms to improve the accuracy of input-entry. In this way, the computing device may accommodate biasing in finger movements that may be specific to a user entering the input

From seen to unseen: Designing keyboard-less interfaces for text entry on the constrained screen real estate of Augmented Reality headsets

Text input is a very challenging task in the constrained screen real-estate of Augmented Reality headsets. Typical keyboards spread over multiple lines and occupy a significant portion of the screen. In this article, we explore the feasibility of single-line text entry systems for smartglasses. We first design FITE, a dynamic keyboard where the characters are positioned depending on their probability within the current input. However, the dynamic layout leads to mediocre text input and low accuracy. We then introduce HIBEY, a fixed 1-line solution that further decreases the screen real-estate usage by hiding the layout. Despite its hidden layout, HIBEY surprisingly performs much better than FITE, and achieves a mean text entry rate of 9.95 words per minute (WPM) with 96.06% accuracy, which is comparable to other state-of-the-art approaches. After 8 days, participants achieve an average of 13.19 WPM. In addition, HIBEY only occupies 13.14% of the screen real estate at the edge region, which is 62.80% smaller than the default keyboard layout on Microsoft Hololens.Peer reviewe

Frame-Based Editing: Easing the Transition from Blocks to Text-Based Programming

Block-based programming systems, such as Scratch or Alice, are the most popular environments for introducing young children to programming. However, mastery of text-based programming continues to be the educational goal for stu- dents who continue to program into their teenage years and beyond. Transitioning across the significant gap between the two editing styles presents a difficult challenge in school- level teaching of programming. We propose a new style of program manipulation to bridge the gap: frame-based edit- ing. Frame-based editing has the resistance to errors and approachability of block-based programming while retaining the flexibility and more conventional programming seman- tics of text-based programming languages. In this paper, we analyse the issues involved in the transition from blocks to text and argue that they can be overcome by using frame- based editing as an intermediate step. A design and imple- mentation of a frame-based editor is provided