Mobile Augmented Reality: User Interfaces, Frameworks, and Intelligence

Mobile Augmented Reality (MAR) integrates computer-generated virtual objects with physical environments for mobile devices. MAR systems enable users to interact with MAR devices, such as smartphones and head-worn wearables, and perform seamless transitions from the physical world to a mixed world with digital entities. These MAR systems support user experiences using MAR devices to provide universal access to digital content. Over the past 20 years, several MAR systems have been developed, however, the studies and design of MAR frameworks have not yet been systematically reviewed from the perspective of user-centric design. This article presents the first effort of surveying existing MAR frameworks (count: 37) and further discuss the latest studies on MAR through a top-down approach: (1) MAR applications; (2) MAR visualisation techniques adaptive to user mobility and contexts; (3) systematic evaluation of MAR frameworks, including supported platforms and corresponding features such as tracking, feature extraction, and sensing capabilities; and (4) underlying machine learning approaches supporting intelligent operations within MAR systems. Finally, we summarise the development of emerging research fields and the current state-of-the-art, and discuss the important open challenges and possible theoretical and technical directions. This survey aims to benefit both researchers and MAR system developers alike.Peer reviewe

Toward cognitive digital twins using a BIM-GIS asset management system for a diffused university

The integrated use of building information modeling (BIM) and geographic information system (GIS) is promising for the development of asset management systems (AMSs) for operation and maintenance (O & M) in smart university campuses. The combination of BIM-GIS with cognitive digital twins (CDTs) can further facilitate the management of complex systems such as university building stock. CDTs enable buildings to behave as autonomous entities, dynamically reacting to environmental changes. Timely decisions based on the actual conditions of buildings and surroundings can be provided, both in emergency scenarios or when optimized and adaptive performances are required. The research aims to develop a BIM-GIS-based AMS for improving user experience and enabling the optimal use of resources in the O & M phase of an Italian university. Campuses are complex assets, mainly diffused with buildings spread across the territory, managed with still document-based and fragmented databases handled by several subjects. This results in incomplete and asymmetrical information, often leading to ineffective and untimely decisions. The paper presents a methodology for the development of a BIM-GIS web-based platform (i.e., AMS-app) providing the real-time visualization of the asset in an interactive 3D map connected to analytical dashboards for management support. Two buildings of the University of Turin are adopted as demonstrators, illustrating the development of an easily accessible, centralized database by integrating spatial and functional data, useful also to develop future CDTs. As a first attempt to show the AMS app potential, crowd simulations have been conducted to understand the buildings' actual level of safety in case of fire emergency and demonstrate how CDTs could improve it. The identification of data needed, also gathered through the future implementation of suitable sensors and Internet of Things networks, is the core issue together with the definition of effective asset visualization and monitoring methods. Future developments will explore the integration of artificial intelligence and immersive technologies to enable space use optimization and real-time wayfinding during evacuation, exploiting digital tools to alert and drive users or authorities for safety improvement. The ability to easily optimize the paths with respect to the actual occupancy and conditions of both the asset and surroundings will be enabled

Partnering People with Deep Learning Systems: Human Cognitive Effects of Explanations

Advances in “deep learning” algorithms have led to intelligent systems that provide automated classifications of unstructured data. Until recently these systems could not provide the reasons behind a classification. This lack of “explainability” has led to resistance in applying these systems in some contexts. An intensive research and development effort to make such systems more transparent and interpretable has proposed and developed multiple types of explanation to address this challenge. Relatively little research has been conducted into how humans process these explanations. Theories and measures from areas of research in social cognition were selected to evaluate attribution of mental processes from intentional systems theory, measures of working memory demands from cognitive load theory, and self-efficacy from social cognition theory. Crowdsourced natural disaster damage assessment of aerial images was employed using a written assessment guideline as the task. The “Wizard of Oz” method was used to generate the damage assessment output of a simulated agent. The output and explanations contained errors consistent with transferring a deep learning system to a new disaster event. A between-subjects experiment was conducted where three types of natural language explanations were manipulated between conditions. Counterfactual explanations increased intrinsic cognitive load and made participants more aware of the challenges of the task. Explanations that described boundary conditions and failure modes (“hedging explanations”) decreased agreement with erroneous agent ratings without a detectable effect on cognitive load. However, these effects were not large enough to counteract decreases in self-efficacy and increases in erroneous agreement as a result of providing a causal explanation. The extraneous cognitive load generated by explanations had the strongest influence on self-efficacy in the task. Presenting all of the explanation types at the same time maximized cognitive load and agreement with erroneous simulated output. Perceived interdependence with the simulated agent was also associated with increases in self-efficacy; however, trust in the agent was not associated with differences in self-efficacy. These findings identify effects related to research areas which have developed methods to design tasks that may increase the effectiveness of explanations

Intelligent Data Analytics using Deep Learning for Data Science

Nowadays, data science stimulates the interest of academics and practitioners because it can assist in the extraction of significant insights from massive amounts of data. From the years 2018 through 2025, the Global Datasphere is expected to rise from 33 Zettabytes to 175 Zettabytes, according to the International Data Corporation. This dissertation proposes an intelligent data analytics framework that uses deep learning to tackle several difficulties when implementing a data science application. These difficulties include dealing with high inter-class similarity, the availability and quality of hand-labeled data, and designing a feasible approach for modeling significant correlations in features gathered from various data sources. The proposed intelligent data analytics framework employs a novel strategy for improving data representation learning by incorporating supplemental data from various sources and structures. First, the research presents a multi-source fusion approach that utilizes confident learning techniques to improve the data quality from many noisy sources. Meta-learning methods based on advanced techniques such as the mixture of experts and differential evolution combine the predictive capacity of individual learners with a gating mechanism, ensuring that only the most trustworthy features or predictions are integrated to train the model. Then, a Multi-Level Convolutional Fusion is presented to train a model on the correspondence between local-global deep feature interactions to identify easily confused samples of different classes. The convolutional fusion is further enhanced with the power of Graph Transformers, aggregating the relevant neighboring features in graph-based input data structures and achieving state-of-the-art performance on a large-scale building damage dataset. Finally, weakly-supervised strategies, noise regularization, and label propagation are proposed to train a model on sparse input labeled data, ensuring the model\u27s robustness to errors and supporting the automatic expansion of the training set. The suggested approaches outperformed competing strategies in effectively training a model on a large-scale dataset of 500k photos, with just about 7% of the images annotated by a human. The proposed framework\u27s capabilities have benefited various data science applications, including fluid dynamics, geometric morphometrics, building damage classification from satellite pictures, disaster scene description, and storm-surge visualization

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

An Overview of Self-Adaptive Technologies Within Virtual Reality Training

This overview presents the current state-of-the-art of self-adaptive technologies within virtual reality (VR) training. Virtual reality training and assessment is increasingly used for five key areas: medical, industrial & commercial training, serious games, rehabilitation and remote training such as Massive Open Online Courses (MOOCs). Adaptation can be applied to five core technologies of VR including haptic devices, stereo graphics, adaptive content, assessment and autonomous agents. Automation of VR training can contribute to automation of actual procedures including remote and robotic assisted surgery which reduces injury and improves accuracy of the procedure. Automated haptic interaction can enable tele-presence and virtual artefact tactile interaction from either remote or simulated environments. Automation, machine learning and data driven features play an important role in providing trainee-specific individual adaptive training content. Data from trainee assessment can form an input to autonomous systems for customised training and automated difficulty levels to match individual requirements. Self-adaptive technology has been developed previously within individual technologies of VR training. One of the conclusions of this research is that while it does not exist, an enhanced portable framework is needed and it would be beneficial to combine automation of core technologies, producing a reusable automation framework for VR training

Technical Workshop: Advanced Helicopter Cockpit Design

Information processing demands on both civilian and military aircrews have increased enormously as rotorcraft have come to be used for adverse weather, day/night, and remote area missions. Applied psychology, engineering, or operational research for future helicopter cockpit design criteria were identified. Three areas were addressed: (1) operational requirements, (2) advanced avionics, and (3) man-system integration

Columbia Chronicle (05/05/1997)

Student newspaper from May 5, 1997 entitled The Chronicle of Columbia College Chicago. This issue is 20 pages and is listed as Volume XXX, Number 25. Cover story: Tuition on the rise again Editor in Chief: John Henry Biedermanhttps://digitalcommons.colum.edu/cadc_chronicle/1381/thumbnail.jp

Modeling Human-Robot Interaction in Three Dimensions

This dissertation answers the question: Can a small autonomous UAV change a person's movements by emulating animal behaviors? Human-robot interaction (HRI) has generally been limited to engagements with ground robots at human height or shorter, essentially working on the same two dimensional plane, but this ignores potential interactions where the robot may be above the human such as small un- manned aerial vehicles (sUAVs) for crowd control and evacuation or for underwater or space vehicles acting as assistants for divers or astronauts. The dissertation combines two approaches {behavioral robotics and HRI {to create a model of \Comfortable Distance" containing the information about human-human and human-ground robot interactions and extends it to three dimensions. Behavioral robotics guides the ex- amination and transfer of relevant behaviors from animals, most notably mammals, birds, and ying insects, into a computational model that can be programmed in simulation and on a sUAV. The validated model of proxemics in three dimensions makes a fundamental contribution to human-robot interaction. The results also have significant benefit to the public safety community, leading to more effective evacuation and crowd control, and possibly saving lives. Three findings from this experiment were important in regards to sUAVs for evacuation: i) expressions focusing on the person, rather than the area, are good for decreasing time (by 7.5 seconds, p <.0001) and preference (by 17.4 %, p <.0001), ii) personal defense behaviors are best for decreasing time of interaction (by about 4 seconds, p <.004), while site defense behaviors are best for increasing distance of interaction (by about .5 m, p <.003), and iii) Hediger's animal zones may be more applicable than Hall's human social zones when considering interactions with animal behaviors in sUAVs