Deep Feature-based Face Detection on Mobile Devices

We propose a deep feature-based face detector for mobile devices to detect user's face acquired by the front facing camera. The proposed method is able to detect faces in images containing extreme pose and illumination variations as well as partial faces. The main challenge in developing deep feature-based algorithms for mobile devices is the constrained nature of the mobile platform and the non-availability of CUDA enabled GPUs on such devices. Our implementation takes into account the special nature of the images captured by the front-facing camera of mobile devices and exploits the GPUs present in mobile devices without CUDA-based frameorks, to meet these challenges.Comment: ISBA 201

A Novel Authentication Method Using Multi-Factor Eye Gaze

A method for novel, rapid and robust one-step multi-factor authentication of a user is presented, employing multi-factor eye gaze. The mobile environment presents challenges that render the conventional password model obsolete. The primary goal is to offer an authentication method that competitively replaces the password, while offering improved security and usability. This method and apparatus combine the smooth operation of biometric authentication with the protection of knowledge based authentication to robustly authenticate a user and secure information on a mobile device in a manner that is easily used and requires no external hardware. This work demonstrates a solution comprised of a pupil segmentation algorithm, gaze estimation, and an innovative application that allows a user to authenticate oneself using gaze as the interaction medium

Driver attention analysis and drowsiness detection using mobile devices

Drowsiness and lack of attention are some of the most fatal and underrated accident causes while driving. In this thesis a non intrusive classifier based on features from drivers' facial movements has been developed, focusing on detection strategies that could be deployed on low-complexity devices, like smartphones. Different classification architectures will be proposed and studied in order to understand which implementation performed the best in terms of detection accuracy.openEmbargo temporaneo per motivi di segretezza e/o di proprietà dei risultati e informazioni di enti esterni o aziende private che hanno partecipato alla realizzazione del lavoro di ricerca relativo alla tes

Emotion detection in real-time on an Android Smartphone

During the last decade the proliferation and capabilities of smartphones have exploded, reaching a situation in which, nowadays, almost everybody carries one of these devices with enormous computational power, cameras and several sensors at any time. The omnipresence, the capabilities and the tools available to facilitate developing applications are the perfect combination for proposing solutions to numerous problems, some existing and some created. In this research, an approach of the widely researched topic of automatically detecting human emotions is brought to the Android platform by means of an application. A system is developed using some existing tools in order to detect emotions with few computations so that it may run in real time on smartphones with less computational power. Given the limited time available for developing this project, the scope of this work is to set the foundation so that the application can be improved in the future by other researchers; therefore, some limitations are set, and some stages are not fully optimized. Finally, future improvements are proposed to facilitate the continuation of this project

Efficient recognition approaches for the interaction between humans and aerial robots

This project consists in a set of computer vision methods that serve as a baseline to perform HRI experiments. The methods are aimed to perform: visual marker detection, face detection and object recognition. Then we tested some of the methods by developing a demonstration scenario

Facial Biometrics on Mobile Devices: Interaction and Quality Assessment

Biometric face recognition is a quick and convenient security method that allows unlocking a smartphone device without the need to remember a PIN code or a password. However, the unconstrained mobile environment brings considerable challenges in facial verification performance. Not only the verification but also the enrolment on the mobile device takes place in unpredictable surroundings. In particular, facial verification involves the enrolment of unsupervised users across a range of environmental conditions, light exposure, and additional variations in terms of user's poses and image background. Is there a way to estimate the variations that a mobile scenario introduces over the facial verification performance? A quality assessment can help in enhancing the biometric performance, but in the context of mobile devices, most of the standardised requirements and methodology presented are based on passport scenarios. A comprehensive analysis should be performed to assess the biometric performance in terms of image quality and user interaction in the particular context of mobile devices. This work aimed to contribute to improving the performance and the adaptability of facial verification systems implemented on smartphones. Fifty-three participants were asked to provide facial images suitable for face verification across several locations and scenarios. A minimum of 150 images per user was collected with a smartphone camera within three different sessions. Sensing data was recorded to assess user interaction during the biometric presentation. Images were also recorded using a Single Lens Reflex camera to enable a comparison with conditions similar to a passport scenario. Results showed the relationship within five selected quality metrics commonly used for quality assessment and the variables introduced by the environment, the user and the camera. Innovative methodologies were also proposed to assess the user interaction using sensors implemented in the smartphone. The analysis underlined important issues and formulated useful observations to enhance facial verification performance on smartphone devices

Visual Techniques for Geological Fieldwork Using Mobile Devices

Visual techniques in general and 3D visualisation in particular have seen considerable adoption within the last 30 years in the geosciences and geology. Techniques such as volume visualisation, for analysing subsurface processes, and photo-coloured LiDAR point-based rendering, to digitally explore rock exposures at the earth’s surface, were applied within geology as one of the first adopting branches of science. A large amount of digital, geological surface- and volume data is nowadays available to desktop-based workflows for geological applications such as hydrocarbon reservoir exploration, groundwater modelling, CO2 sequestration and, in the future, geothermal energy planning. On the other hand, the analysis and data collection during fieldwork has yet to embrace this ”digital revolution”: sedimentary logs, geological maps and stratigraphic sketches are still captured in each geologist’s individual fieldbook, and physical rocks samples are still transported to the lab for subsequent analysis. Is this still necessary, or are there extended digital means of data collection and exploration in the field ? Are modern digital interpretation techniques accurate and intuitive enough to relevantly support fieldwork in geology and other geoscience disciplines ? This dissertation aims to address these questions and, by doing so, close the technological gap between geological fieldwork and office workflows in geology. The emergence of mobile devices and their vast array of physical sensors, combined with touch-based user interfaces, high-resolution screens and digital cameras provide a possible digital platform that can be used by field geologists. Their ubiquitous availability increases the chances to adopt digital workflows in the field without additional, expensive equipment. The use of 3D data on mobile devices in the field is furthered by the availability of 3D digital outcrop models and the increasing ease of their acquisition. This dissertation assesses the prospects of adopting 3D visual techniques and mobile devices within field geology. The research of this dissertation uses previously acquired and processed digital outcrop models in the form of textured surfaces from optical remote sensing and photogrammetry. The scientific papers in this thesis present visual techniques and algorithms to map outcrop photographs in the field directly onto the surface models. Automatic mapping allows the projection of photo interpretations of stratigraphy and sedimentary facies on the 3D textured surface while providing the domain expert with simple-touse, intuitive tools for the photo interpretation itself. The developed visual approach, combining insight from all across the computer sciences dealing with visual information, merits into the mobile device Geological Registration and Interpretation Toolset (GRIT) app, which is assessed on an outcrop analogue study of the Saltwick Formation exposed at Whitby, North Yorkshire, UK. Although being applicable to a diversity of study scenarios within petroleum geology and the geosciences, the particular target application of the visual techniques is to easily provide field-based outcrop interpretations for subsequent construction of training images for multiple point statistics reservoir modelling, as envisaged within the VOM2MPS project. Despite the success and applicability of the visual approach, numerous drawbacks and probable future extensions are discussed in the thesis based on the conducted studies. Apart from elaborating on more obvious limitations originating from the use of mobile devices and their limited computing capabilities and sensor accuracies, a major contribution of this thesis is the careful analysis of conceptual drawbacks of established procedures in modelling, representing, constructing and disseminating the available surface geometry. A more mathematically-accurate geometric description of the underlying algebraic surfaces yields improvements and future applications unaddressed within the literature of geology and the computational geosciences to this date. Also, future extensions to the visual techniques proposed in this thesis allow for expanded analysis, 3D exploration and improved geological subsurface modelling in general.publishedVersio

Translating SIBI (Sign System for Indonesian Gesture) Gesture-to-Text in Real-Time using a Mobile Device

The SIBI gesture translation framework by Rakun was built using a series of machine learning technologies: MobileNetV2 for feature extraction, Conditional Random Field for finding the epenthesis movement frame, and Long Short-Term Memory for word classification. This high computational translation system was previously implemented on a personal computer system, which lacks portability and accessibility. This study implemented the system on a smartphone using an on-device inference method: the translation process is embedded into the smartphone to provide lower latency and zero data usage. The system was then improved using a parallel multi-inference method, which reduced the average translation time by 25%. The final mobile SIBI gesture-to-text translation system achieved a word accuracy of 90.560%, a sentence accuracy of 64%, and an average translation time of 20 seconds

Deep learning systems for estimating visual attention in robot-assisted therapy of children with autism and intellectual disability

Recent studies suggest that some children with autism prefer robots as tutors for improving their social interaction and communication abilities which are impaired due to their disorder. Indeed, research has focused on developing a very promising form of intervention named Robot-Assisted Therapy. This area of intervention poses many challenges, including the necessary flexibility and adaptability to real unconstrained therapeutic settings, which are different from the constrained lab settings where most of the technology is typically tested. Among the most common impairments of children with autism and intellectual disability is social attention, which includes difficulties in establishing the correct visual focus of attention. This article presents an investigation on the use of novel deep learning neural network architectures for automatically estimating if the child is focusing their visual attention on the robot during a therapy session, which is an indicator of their engagement. To study the application, the authors gathered data from a clinical experiment in an unconstrained setting, which provided low-resolution videos recorded by the robot camera during the child–robot interaction. Two deep learning approaches are implemented in several variants and compared with a standard algorithm for face detection to verify the feasibility of estimating the status of the child directly from the robot sensors without relying on bulky external settings, which can distress the child with autism. One of the proposed approaches demonstrated a very high accuracy and it can be used for off-line continuous assessment during the therapy or for autonomously adapting the intervention in future robots with better computational capabilities