Do You Need Instructions Again? Predicting Wayfinding Instruction Demand

The demand for instructions during wayfinding, defined as the frequency of requesting instructions for each decision point, can be considered as an important indicator of the internal cognitive processes during wayfinding. This demand can be a consequence of the mental state of feeling lost, being uncertain, mind wandering, having difficulty following the route, etc. Therefore, it can be of great importance for theoretical cognitive studies on human perception of the environment. From an application perspective, this demand can be used as a measure of the effectiveness of the navigation assistance system. It is therefore worthwhile to be able to predict this demand and also to know what factors trigger it. This paper takes a step in this direction by reporting a successful prediction of instruction demand (accuracy of 78.4%) in a real-world wayfinding experiment with 45 participants, and interpreting the environmental, user, instructional, and gaze-related features that caused it

Data analytics for image visual complexity and kinect-based videos of rehabilitation exercises

With the recent advances in computer vision and pattern recognition, methods from these fields are successfully applied to solve problems in various domains, including health care and social sciences. In this thesis, two such problems, from different domains, are discussed. First, an application of computer vision and broader pattern recognition in physical therapy is presented. Home-based physical therapy is an essential part of the recovery process in which the patient is prescribed specific exercises in order to improve symptoms and daily functioning of the body. However, poor adherence to the prescribed exercises is a common problem. In our work, we explore methods for improving home-based physical therapy experience. We begin by proposing DyAd, a dynamically difficulty adjustment system which captures the trajectory of the hand movement, evaluates the user's performance quantitatively and adjusts the difficulty level for the next trial of the exercise based on the performance measurements. Next, we introduce ExerciseCheck, a remote monitoring and evaluation platform for home-based physical therapy. ExerciseCheck is capable of capturing exercise information, evaluating the performance, providing therapeutic feedback to the patient and the therapist, checking the progress of the user over the course of the physical therapy, and supporting the patient throughout this period. In our experiments, Parkinson patients have tested our system at a clinic and in their homes during their physical therapy period. Our results suggests that ExerciseCheck is a user-friendly application and can assist patients by providing motivation, and guidance to ensure correct execution of the required exercises. As the second application, and within computer vision paradigm, we focus on visual complexity, an image attribute that humans can subjectively evaluate based on the level of details in the image. Visual complexity has been studied in psychophysics, cognitive science, and, more recently, computer vision, for the purposes of product design, web design, advertising, etc. We first introduce a diverse visual complexity dataset which compromises of seven image categories. We collect the ground-truth scores by comparing the pairwise relationship of images and then convert the pairwise scores to absolute scores using mathematical methods. Furthermore, we propose a method to measure the visual complexity that uses unsupervised information extraction from intermediate convolutional layers of deep neural networks. We derive an activation energy metric that combines convolutional layer activations to quantify visual complexity. The high correlations between ground-truth labels and computed energy scores in our experiments show superiority of our method compared to the previous works. Finally, as an example of the relationship between visual complexity and other image attributes, we demonstrate that, within the context of a category, visually more complex images are more memorable to human observers

Visualization and Human-Machine Interaction

The digital age offers a lot of challenges in the eld of visualization. Visual imagery has been effectively used to communicate messages through the ages, to express both abstract and concrete ideas. Today, visualization has ever-expanding applications in science, engineering, education, medicine, entertainment and many other areas. Different areas of research contribute to the innovation in the eld of interactive visualization, such as data science, visual technology, Internet of things and many more. Among them, two areas of renowned importance are Augmented Reality and Visual Analytics. This thesis presents my research in the fields of visualization and human-machine interaction. The purpose of the proposed work is to investigate existing solutions in the area of Augmented Reality (AR) for maintenance. A smaller section of this thesis presents a minor research project on an equally important theme, Visual Analytics. Overall, the main goal is to identify the most important existing problems and then design and develop innovative solutions to address them. The maintenance application domain has been chosen since it is historically one of the first fields of application for Augmented Reality and it offers all the most common and important challenges that AR can arise, as described in chapter 2. Since one of the main problem in AR application deployment is reconfigurability of the application, a framework has been designed and developed that allows the user to create, deploy and update in real-time AR applications. Furthermore, the research focused on the problems related to hand-free interaction, thus investigating the area of speech-recognition interfaces and designing innovative solutions to address the problems of intuitiveness and robustness of the interface. On the other hand, the area of Visual Analytics has been investigated: among the different areas of research, multidimensional data visualization, similarly to AR, poses specific problems related to the interaction between the user and the machine. An analysis of the existing solutions has been carried out in order to identify their limitations and to point out possible improvements. Since this analysis delineates the scatterplot as a renowned visualization tool worthy of further research, different techniques for adapting its usage to multidimensional data are analyzed. A multidimensional scatterplot has been designed and developed in order to perform a comparison with another multidimensional visualization tool, the ScatterDice. The first chapters of my thesis describe my investigations in the area of Augmented Reality for maintenance. Chapter 1 provides definitions for the most important terms and an introduction to AR. The second chapter focuses on maintenance, depicting the motivations that led to choose this application domain. Moreover, the analysis concerning open problems and related works is described along with the methodology adopted to design and develop the proposed solutions. The third chapter illustrates how the adopted methodology has been applied in order to assess the problems described in the previous one. Chapter 4 describes the methodology adopted to carry out the tests and outlines the experimental results, whereas the fifth chapter illustrates the conclusions and points out possible future developments. Chapter 6 describes the analysis and research work performed in the eld of Visual Analytics, more specifically on multidimensional data visualizations. Overall, this thesis illustrates how the proposed solutions address common problems of visualization and human-machine interaction, such as interface de- sign, robustness of the interface and acceptance of new technology, whereas other problems are related to the specific research domain, such as pose tracking and reconfigurability of the procedure for the AR domain

A survey of usability issues in mobile map-based systems

Dissertation submitted in partial fulfilment of the requirements for the Degree of Master of Science in Geospatial TechnologiesHow geospatial information could be represented in map or other forms of communication to display in mobile phones to convey spatial knowledge to users more effective and efficient with less ambiguity? This triggering question stems from the usability problems available in mobile map-based systems, that made using mobile navigation services and applications for pedestrians, a tedious and complicated task which is rather confusing to be helpful. Problems such as; losing the spatial overview of the area, overload of information in small screens of mobile phones, visibility issue of off-screen entities, weaknesses in orienting users with real environment, too much engagement of users with interface which causes environment distraction and so on. There are a lot of solutions have proposed to mitigate these available issues in mobile map-based systems, but each one has its pros and cons that is not complete enough to tackle above mentioned issues alone, and most of the time a combination of them is proposing. We tried with systematic literature review (SLR) that is more reliable, replicable and valid [1], find the most frequently applied usability evaluation method in the available studies to detect the usability issues in mobile map-based systems (MMSs), then find the most frequently usability issues that detected among the reviewed literatures and how to categorize them, in what contexts they mostly happened and what solutions proposed so far to resolve them. We operated tree iterations of systematic literature review (SLR) with totally 8667 identified publications (within 6 relevant databases and a search engine with priority of 4 most prominent and relevant journals and conferences in the field of mobile HCI and location based services), that 196 one of them included in first screening1, were thoroughly read in order to check with predefined inclusion criteria and overall, 56 papers (between those 196 papers) that qualified with our well-defined and updated inclusion criteria properties read in-depth at least two times to extract the data. In the first iteration 25 papers have reviewed and relevant data with considering our research questions has extracted and reflected in the first iteration table. In the second iteration, 24 papers which had adjusted inclusion criteria parameters have included to data extraction for filling the updated table. The last iteration according to the scarcity of publications in this realm and time limitation, has operated only with 7 literatures and relevant data extracted to fill in the last updated table. Results of the SLR showed the most frequently usability evaluation method was “Questionnaire” to achieve effectiveness and efficiency of the system, and the most frequently usability issue that detected within available literatures was “losing the spatial overview” which followed by “too much zooming and panning operations by users” that stems from the same problem; small screen size of mobile devices. We categorized the issues into two main groups of technological and spatial issues, which we only here focused on the usability issues relevant to map interfaces in mobile phones (spatial issues), not the technological problems relevant to the server or the hardware perspective (sensors, connectivity, battery drainage, GPS accuracy etc.). We have noticed the most frequently usability issue has happened in the mobile phone with average screen size of 3.83 inches, 87% of the cases in the laboratory environment, with users (not experts) with average age of 26 years old that 64.2% of them had relevant knowledge (GI2 knowledge). The low amount of field-based studies highlights the lack of considering real context in available case studies that in usability evaluation of location based mobile systems is highly important. Some traditional solutions have proposed to address the most frequently occurred usability problem in mobile map-based systems such as the techniques for visualizing the off-screen objects (such as Overview&Detail, Scaled Arrows, Wedge etc.) and some techniques for enhancing the zoom and pan operations (such as vario-scale maps, semi-automatic zooming (SAZ), tilt zooming, content zooming, anchored zoom etc.) that none of them were not completely suitable enough to be applied in these systems and the most famous systems such as Google Maps still working without taking advantage of such approaches, techniques and widgets, with a lot of usability issues

Human-Machine Interfaces for Service Robotics

L'abstract è presente nell'allegato / the abstract is in the attachmen