Login Authentication with Facial Gesture Recognition

Facial recognition has proven to be very useful and versatile, from Facebook photo tagging and Snapchat filters to modeling fluid dynamics and designing for augmented reality. However, facial recognition has only been used for user login services in conjunction with expensive and restrictive hardware technologies, such as in smart phone devices like the iPhone x. This project aims to apply machine learning techniques to reliably distinguish user accounts with only common cameras to make facial recognition logins more accessible to website and software developers. To show the feasibility of this idea, we created a web API that recognizes a users face to log them in to their account, and we will create a simple website to test the reliability of our system. In this paper, we discuss our database-centric architecture model, use cases and activity diagrams, technologies we used for the website, API, and machine learning algorithms. We also provide the screenshots of our system, the user manual, and our future plan

GTmoPass: Two-factor Authentication on Public Displays Using Gaze-touch Passwords and Personal Mobile Devices

As public displays continue to deliver increasingly private and personalized content, there is a need to ensure that only the legitimate users can access private information in sensitive contexts. While public displays can adopt similar authentication concepts like those used on public terminals (e.g., ATMs), authentication in public is subject to a number of risks. Namely, adversaries can uncover a user's password through (1) shoulder surfing, (2) thermal attacks, or (3) smudge attacks. To address this problem we propose GTmoPass, an authentication architecture that enables Multi-factor user authentication on public displays. The first factor is a knowledge-factor: we employ a shoulder-surfing resilient multimodal scheme that combines gaze and touch input for password entry. The second factor is a possession-factor: users utilize their personal mobile devices, on which they enter the password. Credentials are securely transmitted to a server via Bluetooth beacons. We describe the implementation of GTmoPass and report on an evaluation of its usability and security, which shows that although authentication using GTmoPass is slightly slower than traditional methods, it protects against the three aforementioned threats

GazeTouchPIN: Protecting Sensitive Data on Mobile Devices Using Secure Multimodal Authentication

Although mobile devices provide access to a plethora of sensitive data, most users still only protect them with PINs or patterns, which are vulnerable to side-channel attacks (e.g., shoulder surfing). How-ever, prior research has shown that privacy-aware users are willing to take further steps to protect their private data. We propose GazeTouchPIN, a novel secure authentication scheme for mobile devices that combines gaze and touch input. Our multimodal approach complicates shoulder-surfing attacks by requiring attackers to ob-serve the screen as well as the user’s eyes to and the password. We evaluate the security and usability of GazeTouchPIN in two user studies (N=30). We found that while GazeTouchPIN requires longer entry times, privacy aware users would use it on-demand when feeling observed or when accessing sensitive data. The results show that successful shoulder surfing attack rate drops from 68% to 10.4%when using GazeTouchPIN

Understanding face and eye visibility in front-facing cameras of smartphones used in the wild

Commodity mobile devices are now equipped with high-resolution front-facing cameras, allowing applications in biometrics (e.g., FaceID in the iPhone X), facial expression analysis, or gaze interaction. However, it is unknown how often users hold devices in a way that allows capturing their face or eyes, and how this impacts detection accuracy. We collected 25,726 in-the-wild photos, taken from the front-facing camera of smartphones as well as associated application usage logs. We found that the full face is visible about 29% of the time, and that in most cases the face is only partially visible. Furthermore, we identified an influence of users' current activity; for example, when watching videos, the eyes but not the entire face are visible 75% of the time in our dataset. We found that a state-of-the-art face detection algorithm performs poorly against photos taken from front-facing cameras. We discuss how these findings impact mobile applications that leverage face and eye detection, and derive practical implications to address state-of-the art's limitations

Risks and potentials of graphical and gesture-based authentication for touchscreen mobile devices

While a few years ago, mobile phones were mainly used for making phone calls and texting short messages, the functionality of mobile devices has massively grown. We are surfing the web, sending emails and we are checking our bank accounts on the go. As a consequence, these internet-enabled devices store a lot of potentially sensitive data and require enhanced protection. We argue that authentication often represents the only countermeasure to protect mobile devices from unwanted access. Knowledge-based concepts (e.g., PIN) are the most used authentication schemes on mobile devices. They serve as the main protection barrier for many users and represent the fallback solution whenever alternative mechanisms fail (e.g., fingerprint recognition). This thesis focuses on the risks and potentials of gesture-based authentication concepts that particularly exploit the touch feature of mobile devices. The contribution of our work is threefold. Firstly, the problem space of mobile authentication is explored. Secondly, the design space is systematically evaluated utilizing interactive prototypes. Finally, we provide generalized insights into the impact of specific design factors and present recommendations for the design and the evaluation of graphical gesture-based authentication mechanisms. The problem space exploration is based on four research projects that reveal important real-world issues of gesture-based authentication on mobile devices. The first part focuses on authentication behavior in the wild and shows that the mobile context makes great demands on the usability of authentication concepts. The second part explores usability features of established concepts and indicates that gesture-based approaches have several benefits in the mobile context. The third part focuses on observability and presents a prediction model for the vulnerability of a given grid-based gesture. Finally, the fourth part investigates the predictability of user-selected gesture-based secrets. The design space exploration is based on a design-oriented research approach and presents several practical solutions to existing real-world problems. The novel authentication mechanisms are implemented into working prototypes and evaluated in the lab and the field. In the first part, we discuss smudge attacks and present alternative authentication concepts that are significantly more secure against such attacks. The second part focuses on observation attacks. We illustrate how relative touch gestures can support eyes-free authentication and how they can be utilized to make traditional PIN-entry secure against observation attacks. The third part addresses the problem of predictable gesture choice and presents two concepts which nudge users to select a more diverse set of gestures. Finally, the results of the basic research and the design-oriented applied research are combined to discuss the interconnection of design space and problem space. We contribute by outlining crucial requirements for mobile authentication mechanisms and present empirically proven objectives for future designs. In addition, we illustrate a systematic goal-oriented development process and provide recommendations for the evaluation of authentication on mobile devices.Während Mobiltelefone vor einigen Jahren noch fast ausschließlich zum Telefonieren und zum SMS schreiben genutzt wurden, sind die Anwendungsmöglichkeiten von Mobilgeräten in den letzten Jahren erheblich gewachsen. Wir surfen unterwegs im Netz, senden E-Mails und überprüfen Bankkonten. In der Folge speichern moderne internetfähigen Mobilgeräte eine Vielfalt potenziell sensibler Daten und erfordern einen erhöhten Schutz. In diesem Zusammenhang stellen Authentifizierungsmethoden häufig die einzige Möglichkeit dar, um Mobilgeräte vor ungewolltem Zugriff zu schützen. Wissensbasierte Konzepte (bspw. PIN) sind die meistgenutzten Authentifizierungssysteme auf Mobilgeräten. Sie stellen für viele Nutzer den einzigen Schutzmechanismus dar und dienen als Ersatzlösung, wenn alternative Systeme (bspw. Fingerabdruckerkennung) versagen. Diese Dissertation befasst sich mit den Risiken und Potenzialen gestenbasierter Konzepte, welche insbesondere die Touch-Funktion moderner Mobilgeräte ausschöpfen. Der wissenschaftliche Beitrag dieser Arbeit ist vielschichtig. Zum einen wird der Problemraum mobiler Authentifizierung erforscht. Zum anderen wird der Gestaltungsraum anhand interaktiver Prototypen systematisch evaluiert. Schließlich stellen wir generelle Einsichten bezüglich des Einflusses bestimmter Gestaltungsaspekte dar und geben Empfehlungen für die Gestaltung und Bewertung grafischer gestenbasierter Authentifizierungsmechanismen. Die Untersuchung des Problemraums basiert auf vier Forschungsprojekten, welche praktische Probleme gestenbasierter Authentifizierung offenbaren. Der erste Teil befasst sich mit dem Authentifizierungsverhalten im Alltag und zeigt, dass der mobile Kontext hohe Ansprüche an die Benutzerfreundlichkeit eines Authentifizierungssystems stellt. Der zweite Teil beschäftigt sich mit der Benutzerfreundlichkeit etablierter Methoden und deutet darauf hin, dass gestenbasierte Konzepte vor allem im mobilen Bereich besondere Vorzüge bieten. Im dritten Teil untersuchen wir die Beobachtbarkeit gestenbasierter Eingabe und präsentieren ein Vorhersagemodell, welches die Angreifbarkeit einer gegebenen rasterbasierten Geste abschätzt. Schließlich beschäftigen wir uns mit der Erratbarkeit nutzerselektierter Gesten. Die Untersuchung des Gestaltungsraums basiert auf einem gestaltungsorientierten Forschungsansatz, welcher zu mehreren praxisgerechte Lösungen führt. Die neuartigen Authentifizierungskonzepte werden als interaktive Prototypen umgesetzt und in Labor- und Feldversuchen evaluiert. Im ersten Teil diskutieren wir Fettfingerattacken ("smudge attacks") und präsentieren alternative Authentifizierungskonzepte, welche effektiv vor diesen Angriffen schützen. Der zweite Teil beschäftigt sich mit Angriffen durch Beobachtung und verdeutlicht wie relative Gesten dazu genutzt werden können, um blickfreie Authentifizierung zu gewährleisten oder um PIN-Eingaben vor Beobachtung zu schützen. Der dritte Teil beschäftigt sich mit dem Problem der vorhersehbaren Gestenwahl und präsentiert zwei Konzepte, welche Nutzer dazu bringen verschiedenartige Gesten zu wählen. Die Ergebnisse der Grundlagenforschung und der gestaltungsorientierten angewandten Forschung werden schließlich verknüpft, um die Verzahnung von Gestaltungsraum und Problemraum zu diskutieren. Wir präsentieren wichtige Anforderungen für mobile Authentifizierungsmechanismen und erläutern empirisch nachgewiesene Zielvorgaben für zukünftige Konzepte. Zusätzlich zeigen wir einen zielgerichteten Entwicklungsprozess auf, welcher bei der Entwicklung neuartiger Konzepte helfen wird und geben Empfehlungen für die Evaluation mobiler Authentifizierungsmethoden

Forgery-Resistant Touch-based Authentication on Mobile Devices

Mobile devices store a diverse set of private user data and have gradually become a hub to control users' other personal Internet-of-Things devices. Access control on mobile devices is therefore highly important. The widely accepted solution is to protect access by asking for a password. However, password authentication is tedious, e.g., a user needs to input a password every time she wants to use the device. Moreover, existing biometrics such as face, fingerprint, and touch behaviors are vulnerable to forgery attacks. We propose a new touch-based biometric authentication system that is passive and secure against forgery attacks. In our touch-based authentication, a user's touch behaviors are a function of some random "secret". The user can subconsciously know the secret while touching the device's screen. However, an attacker cannot know the secret at the time of attack, which makes it challenging to perform forgery attacks even if the attacker has already obtained the user's touch behaviors. We evaluate our touch-based authentication system by collecting data from 25 subjects. Results are promising: the random secrets do not influence user experience and, for targeted forgery attacks, our system achieves 0.18 smaller Equal Error Rates (EERs) than previous touch-based authentication.Comment: Accepted for publication by ASIACCS'1

Orientation Based Accelerometer Analysis (OBAA) for Mobile Gestures: Memorable Authentication

Mobile authentication today primarily relies onPersonal Identification Numbers (PINs). For PINs to be securefrom the majority of malicious users, it must contain a highnumber of digits and be entropic. Human memory generallystruggles when it attempts to recall highly entropic numericcodes. Gesture-based authentication using Quick Reference (QR)codes, and internally analyzed accelerometer data from mobiledevices, allow for sustaining a more user-friendly, memorable,and low expense alternative to PINs. This paper presents atechnique for users to capture movements of their mobile deviceby analyzing the orientation of devices and the speed at whichthese orientations transition via accelerometer data. Thesemotions are described as the user’s gesture. Gestures can be usedto identify a user, while QR codes can be used to indicate aspecific machine a user can attempt to authenticate with. A userstudy was performed and showed gesture-based authenticationresults in a more user preferred, entropic and memorableauthentication system in comparison to similar applications