Towards Naturalistic Interfaces of Virtual Reality Systems

Interaction plays a key role in achieving realistic experience in virtual reality (VR). Its realization depends on interpreting the intents of human motions to give inputs to VR systems. Thus, understanding human motion from the computational perspective is essential to the design of naturalistic interfaces for VR. This dissertation studied three types of human motions, including locomotion (walking), head motion and hand motion in the context of VR. For locomotion, the dissertation presented a machine learning approach for developing a mechanical repositioning technique based on a 1-D treadmill for interacting with a unique new large-scale projective display, called the Wide-Field Immersive Stereoscopic Environment (WISE). The usability of the proposed approach was assessed through a novel user study that asked participants to pursue a rolling ball at variable speed in a virtual scene. In addition, the dissertation studied the role of stereopsis in avoiding virtual obstacles while walking by asking participants to step over obstacles and gaps under both stereoscopic and non-stereoscopic viewing conditions in VR experiments. In terms of head motion, the dissertation presented a head gesture interface for interaction in VR that recognizes real-time head gestures on head-mounted displays (HMDs) using Cascaded Hidden Markov Models. Two experiments were conducted to evaluate the proposed approach. The first assessed its offline classification performance while the second estimated the latency of the algorithm to recognize head gestures. The dissertation also conducted a user study that investigated the effects of visual and control latency on teleoperation of a quadcopter using head motion tracked by a head-mounted display. As part of the study, a method for objectively estimating the end-to-end latency in HMDs was presented. For hand motion, the dissertation presented an approach that recognizes dynamic hand gestures to implement a hand gesture interface for VR based on a static head gesture recognition algorithm. The proposed algorithm was evaluated offline in terms of its classification performance. A user study was conducted to compare the performance and the usability of the head gesture interface, the hand gesture interface and a conventional gamepad interface for answering Yes/No questions in VR. Overall, the dissertation has two main contributions towards the improvement of naturalism of interaction in VR systems. Firstly, the interaction techniques presented in the dissertation can be directly integrated into existing VR systems offering more choices for interaction to end users of VR technology. Secondly, the results of the user studies of the presented VR interfaces in the dissertation also serve as guidelines to VR researchers and engineers for designing future VR systems

Indoor navigation for the visually impaired : enhancements through utilisation of the Internet of Things and deep learning

Wayfinding and navigation are essential aspects of independent living that heavily rely on the sense of vision. Walking in a complex building requires knowing exact location to find a suitable path to the desired destination, avoiding obstacles and monitoring orientation and movement along the route. People who do not have access to sight-dependent information, such as that provided by signage, maps and environmental cues, can encounter challenges in achieving these tasks independently. They can rely on assistance from others or maintain their independence by using assistive technologies and the resources provided by smart environments. Several solutions have adapted technological innovations to combat navigation in an indoor environment over the last few years. However, there remains a significant lack of a complete solution to aid the navigation requirements of visually impaired (VI) people. The use of a single technology cannot provide a solution to fulfil all the navigation difficulties faced. A hybrid solution using Internet of Things (IoT) devices and deep learning techniques to discern the patterns of an indoor environment may help VI people gain confidence to travel independently. This thesis aims to improve the independence and enhance the journey of VI people in an indoor setting with the proposed framework, using a smartphone. The thesis proposes a novel framework, Indoor-Nav, to provide a VI-friendly path to avoid obstacles and predict the user s position. The components include Ortho-PATH, Blue Dot for VI People (BVIP), and a deep learning-based indoor positioning model. The work establishes a novel collision-free pathfinding algorithm, Orth-PATH, to generate a VI-friendly path via sensing a grid-based indoor space. Further, to ensure correct movement, with the use of beacons and a smartphone, BVIP monitors the movements and relative position of the moving user. In dark areas without external devices, the research tests the feasibility of using sensory information from a smartphone with a pre-trained regression-based deep learning model to predict the user s absolute position. The work accomplishes a diverse range of simulations and experiments to confirm the performance and effectiveness of the proposed framework and its components. The results show that Indoor-Nav is the first type of pathfinding algorithm to provide a novel path to reflect the needs of VI people. The approach designs a path alongside walls, avoiding obstacles, and this research benchmarks the approach with other popular pathfinding algorithms. Further, this research develops a smartphone-based application to test the trajectories of a moving user in an indoor environment

Classification of non-heat generating outdoor objects in thermal scenes for autonomous robots

We have designed and implemented a physics-based adaptive Bayesian pattern classification model that uses a passive thermal infrared imaging system to automatically characterize non-heat generating objects in unstructured outdoor environments for mobile robots. In the context of this research, non-heat generating objects are defined as objects that are not a source for their own emission of thermal energy, and so exclude people, animals, vehicles, etc. The resulting classification model complements an autonomous bot\u27s situational awareness by providing the ability to classify smaller structures commonly found in the immediate operational environment. Since GPS depends on the availability of satellites and onboard terrain maps which are often unable to include enough detail for smaller structures found in an operational environment, bots will require the ability to make decisions such as go through the hedges or go around the brick wall. A thermal infrared imaging modality mounted on a small mobile bot is a favorable choice for receiving enough detailed information to automatically interpret objects at close ranges while unobtrusively traveling alongside pedestrians. The classification of indoor objects and heat generating objects in thermal scenes is a solved problem. A missing and essential piece in the literature has been research involving the automatic characterization of non-heat generating objects in outdoor environments using a thermal infrared imaging modality for mobile bots. Seeking to classify non-heat generating objects in outdoor environments using a thermal infrared imaging system is a complex problem due to the variation of radiance emitted from the objects as a result of the diurnal cycle of solar energy. The model that we present will allow bots to see beyond vision to autonomously assess the physical nature of the surrounding structures for making decisions without the need for an interpretation by humans.;Our approach is an application of Bayesian statistical pattern classification where learning involves labeled classes of data (supervised classification), assumes no formal structure regarding the density of the data in the classes (nonparametric density estimation), and makes direct use of prior knowledge regarding an object class\u27s existence in a bot\u27s immediate area of operation when making decisions regarding class assignments for unknown objects. We have used a mobile bot to systematically capture thermal infrared imagery for two categories of non-heat generating objects (extended and compact) in several different geographic locations. The extended objects consist of objects that extend beyond the thermal camera\u27s field of view, such as brick walls, hedges, picket fences, and wood walls. The compact objects consist of objects that are within the thermal camera\u27s field of view, such as steel poles and trees. We used these large representative data sets to explore the behavior of thermal-physical features generated from the signals emitted by the classes of objects and design our Adaptive Bayesian Classification Model. We demonstrate that our novel classification model not only displays exceptional performance in characterizing non-heat generating outdoor objects in thermal scenes but it also outperforms the traditional KNN and Parzen classifiers

Development of a simulation tool for measurements and analysis of simulated and real data to identify ADLs and behavioral trends through statistics techniques and ML algorithms

openCon una popolazione di anziani in crescita, il numero di soggetti a rischio di patologia è in rapido aumento. Molti gruppi di ricerca stanno studiando soluzioni pervasive per monitorare continuamente e discretamente i soggetti fragili nelle loro case, riducendo i costi sanitari e supportando la diagnosi medica. Comportamenti anomali durante l'esecuzione di attività di vita quotidiana (ADL) o variazioni sulle tendenze comportamentali sono di grande importanza.With a growing population of elderly people, the number of subjects at risk of pathology is rapidly increasing. Many research groups are studying pervasive solutions to continuously and unobtrusively monitor fragile subjects in their homes, reducing health-care costs and supporting the medical diagnosis. Anomalous behaviors while performing activities of daily living (ADLs) or variations on behavioral trends are of great importance. To measure ADLs a significant number of parameters need to be considering affecting the measurement such as sensors and environment characteristics or sensors disposition. To face the impossibility to study in the real context the best configuration of sensors able to minimize costs and maximize accuracy, simulation tools are being developed as powerful means. This thesis presents several contributions on this topic. In the following research work, a study of a measurement chain aimed to measure ADLs and represented by PIRs sensors and ML algorithm is conducted and a simulation tool in form of Web Application has been developed to generate datasets and to simulate how the measurement chain reacts varying the configuration of the sensors. Starting from eWare project results, the simulation tool has been thought to provide support for technicians, developers and installers being able to speed up analysis and monitoring times, to allow rapid identification of changes in behavioral trends, to guarantee system performance monitoring and to study the best configuration of the sensors network for a given environment. The UNIVPM Home Care Web App offers the chance to create ad hoc datasets related to ADLs and to conduct analysis thanks to statistical algorithms applied on data. To measure ADLs, machine learning algorithms have been implemented in the tool. Five different tasks have been identified. To test the validity of the developed instrument six case studies divided into two categories have been considered. To the first category belong those studies related to: 1) discover the best configuration of the sensors keeping environmental characteristics and user behavior as constants; 2) define the most performant ML algorithms. The second category aims to proof the stability of the algorithm implemented and its collapse condition by varying user habits. Noise perturbation on data has been applied to all case studies. Results show the validity of the generated datasets. By maximizing the sensors network is it possible to minimize the ML error to 0.8%. Due to cost is a key factor in this scenario, the fourth case studied considered has shown that minimizing the configuration of the sensors it is possible to reduce drastically the cost with a more than reasonable value for the ML error around 11.8%. Results in ADLs measurement can be considered more than satisfactory.INGEGNERIA INDUSTRIALEopenPirozzi, Michel

Smart aging : utilisation of machine learning and the Internet of Things for independent living

Smart aging utilises innovative approaches and technology to improve older adults’ quality of life, increasing their prospects of living independently. One of the major concerns the older adults to live independently is “serious fall”, as almost a third of people aged over 65 having a fall each year. Dementia, affecting nearly 9% of the same age group, poses another significant issue that needs to be identified as early as possible. Existing fall detection systems from the wearable sensors generate many false alarms; hence, a more accurate and secure system is necessary. Furthermore, there is a considerable gap to identify the onset of cognitive impairment using remote monitoring for self-assisted seniors living in their residences. Applying biometric security improves older adults’ confidence in using IoT and makes it easier for them to benefit from smart aging. Several publicly available datasets are pre-processed to extract distinctive features to address fall detection shortcomings, identify the onset of dementia system, and enable biometric security to wearable sensors. These key features are used with novel machine learning algorithms to train models for the fall detection system, identifying the onset of dementia system, and biometric authentication system. Applying a quantitative approach, these models are tested and analysed from the test dataset. The fall detection approach proposed in this work, in multimodal mode, can achieve an accuracy of 99% to detect a fall. Additionally, using 13 selected features, a system for detecting early signs of dementia is developed. This system has achieved an accuracy rate of 93% to identify a cognitive decline in the older adult, using only some selected aspects of their daily activities. Furthermore, the ML-based biometric authentication system uses physiological signals, such as ECG and Photoplethysmogram, in a fusion mode to identify and authenticate a person, resulting in enhancement of their privacy and security in a smart aging environment. The benefits offered by the fall detection system, early detection and identifying the signs of dementia, and the biometric authentication system, can improve the quality of life for the seniors who prefer to live independently or by themselves

Mindful navigation with guiding light : design considerations for projector based indoor navigation assistance system

Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 113-117).People can easily become mindless in their decision-making and become disengaged from their surroundings when their actions depend on information and guidance from an assistive technology. Research has shown how automated navigation assistance systems lead users to be disengaged from the space through which they are traveling, resulting in poor recollection of the environment and poorer situational decision-making. This disengagement and mindlessness can potentially increase the risk of accidents and lower the quality of user experience. If we can help people become mindfully attentive to the environment and surroundings while carrying out navigation tasks using assistive technologies, I hypothesize that we will have better memory of the space, improved cognitive reconstruction of environment, and better understanding of the immediate situation, all of which will lead to better decision making and more efficient navigation. In this work, I present a new approach for analyzing the problem of navigation assistance for pedestrians, which considers both the physical and psychological constraints of users focused on navigation. I address the physical constraint that eyes should remain "on the street" by providing a new visual interface, named Guiding Light, that offers a mixed reality presentation of guidance information in the environment itself, instead of on a screen. We address the psychological constraint that minds should remain engaged with the environment by applying a framework based on mindfulness and mindlessness theory (Langer 1989) in the design of the system. The theory explains how mindsets affect engagement levels and decision-making in daily activities. In addition, this thesis describes an indoor positioning technology that provides relatively high accuracy localization and heading orientation of a user in indoor environments. The innovation not only involved developing a new sensor but also a software system to collect fingerprint maps and tracking location with the fingerprint maps. This new technology opens up a new area in the field to explore other possibilities of using a magnetic field based positioning system.by Jaewoo Chung.Ph.D

Device-free indoor localisation with non-wireless sensing techniques : a thesis by publications presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Electronics and Computer Engineering, Massey University, Albany, New Zealand

Global Navigation Satellite Systems provide accurate and reliable outdoor positioning to support a large number of applications across many sectors. Unfortunately, such systems do not operate reliably inside buildings due to the signal degradation caused by the absence of a clear line of sight with the satellites. The past two decades have therefore seen intensive research into the development of Indoor Positioning System (IPS). While considerable progress has been made in the indoor localisation discipline, there is still no widely adopted solution. The proliferation of Internet of Things (IoT) devices within the modern built environment provides an opportunity to localise human subjects by utilising such ubiquitous networked devices. This thesis presents the development, implementation and evaluation of several passive indoor positioning systems using ambient Visible Light Positioning (VLP), capacitive-flooring, and thermopile sensors (low-resolution thermal cameras). These systems position the human subject in a device-free manner (i.e., the subject is not required to be instrumented). The developed systems improve upon the state-of-the-art solutions by offering superior position accuracy whilst also using more robust and generalised test setups. The developed passive VLP system is one of the first reported solutions making use of ambient light to position a moving human subject. The capacitive-floor based system improves upon the accuracy of existing flooring solutions as well as demonstrates the potential for automated fall detection. The system also requires very little calibration, i.e., variations of the environment or subject have very little impact upon it. The thermopile positioning system is also shown to be robust to changes in the environment and subjects. Improvements are made over the current literature by testing across multiple environments and subjects whilst using a robust ground truth system. Finally, advanced machine learning methods were implemented and benchmarked against a thermopile dataset which has been made available for other researchers to use

A methodology for developing Second Life environments using case-based reasoning techniques

Launched in 2003, Second Life is a computer-based pseudo-environment accessed via the Internet. Although a number of individuals and companies have developed a presence (lands) in Second Life, no appropriate methodology has been put into place for undertaking such developments. Although users have adapted existing methods to their individual needs, this research project explores the development of a methodology for developing lands specifically within Second Life. After researching and examining a variety of different software methods and techniques, it was decided to base this research project methodology on Case-Based Reasoning (CBR) techniques, which shares a number of synergies with Second Life itself. With some modifications, a web-based system was designed based on CBR to work in accordance with Second Life. Collecting and analyzing the feedback for the first version of the web-based system identified the adjustments and improvements needed. Therefore, from tracking its progress against previous specifications and future activity, an updated version of the CBR web-based system covering the latest changes and improvements of the tool was introduced. In addition to this, new functionalities have been added in the improved version in order to refine and develop the original prototype to become a highly effective SL development tool. New feedback platforms have been provided to facilitate the use of the system and to obtain results which are more closely related to the users recommendations. Through the feedback process, the tool is becoming ever more useful to developers of Second Life systems. This research project discusses the use of Case-based reasoning techniques and evaluates their application to the development of space within Second Life