Multidimensional Pareto optimization of touchscreen keyboards for speed, familiarity and improved spell checking

The paper presents a new optimization technique for keyboard layouts based on Pareto front optimization. We used this multifactorial technique to create two new touchscreen phone keyboard layouts based on three design metrics: minimizing finger travel distance in order to maximize text entry speed, a new metric to maximize the quality of spell correction quality by minimizing neighbouring key ambiguity, and maximizing familiarity through a similarity function with the standard Qwerty layout. The paper describes the optimization process and resulting layouts for a standard trapezoid shaped keyboard and a more rectangular layout. Fitts' law modelling shows a predicted 11% improvement in entry speed without taking into account the significantly improved error correction potential and the subsequent effect on speed. In initial user tests typing speed dropped from approx. 21wpm with Qwerty to 13wpm (64%) on first use of our layout but recovered to 18wpm (85%) within four short trial sessions, and was still improving. NASA TLX forms showed no significant difference on load between Qwerty and our new layout use in the fourth session. Together we believe this shows the new layouts are faster and can be quickly adopted by users

Reinforcement learning and planning for autonomous agent navigation:With a focus on sparse reward settings

Being able to navigate our surroundings enables us humans to freely interact with our environment and is therefore an important skill for truly autonomous technical systems as well. The machine learning paradigm of reinforcement learning (RL) enables learning (neural network) policies for decision making through continuous interaction with the environment. However, if the rewards that are received as feedback are sparse, improving the policy gets difficult and inefficient. Therefore, this thesis focusses on improving policy learning under sparse rewards for autonomous agents tasked to reach dedicated goal locations.First, we present a novel spatial gradient (SG) strategy to select starting states at the boundary of the agents’ capabilities, which results in a curriculum that improves learning progress.Afterwards, we combine planning over abstract sub-goals with reinforcement learning to obtain policies to reach these sub-goals. The resulting sub-tasks make policy learning easier.We first present our hierarchical VI-RL policy architecture that utilizes a learned transition model for planning, which captures agent capabilities and enables generalization.Subsequently, we improve efficiency and performance of the sub-goal planning by learning to locally refine simple shortest path plans based on detailed local state information. Our proposed RL-trained Value Refinement Network (VRN) architecture additionally enables navigating dynamic environments without repeated global re-planning.Finally, we address the practically relevant setting where continuous environment interaction is not possible. Our HORIBLe-VRN algorithm allows to learn our hierarchical planning-based policies from pre-collected data, incorporating latent sub-goal inference as well as offline RL to improve over sub-optimal demonstrations

Simulation in Automated Guided Vehicle System Design

The intense global competition that manufacturing companies face today results in an increase of product variety and shorter product life cycles. One response to this threat is agile manufacturing concepts. This requires materials handling systems that are agile and capable of reconfiguration. As competition in the world marketplace becomes increasingly customer-driven, manufacturing environments must be highly reconfigurable and responsive to accommodate product and process changes, with rigid, static automation systems giving way to more flexible types. Automated Guided Vehicle Systems (AGVS) have such capabilities and AGV functionality has been developed to improve flexibility and diminish the traditional disadvantages of AGV-systems. The AGV-system design is however a multi-faceted problem with a large number of design factors of which many are correlating and interdependent. Available methods and techniques exhibit problems in supporting the whole design process. A research review of the work reported on AGVS development in combination with simulation revealed that of 39 papers only four were industrially related. Most work was on the conceptual design phase, but little has been reported on the detailed simulation of AGVS. Semi-autonomous vehicles (SA V) are an innovative concept to overcome the problems of inflexible -systems and to improve materials handling functionality. The SA V concept introduces a higher degree of autonomy in industrial AGV -systems with the man-in-the-Ioop. The introduction of autonomy in industrial applications is approached by explicitly controlling the level of autonomy at different occasions. The SA V s are easy to program and easily reconfigurable regarding navigation systems and material handling equipment. Novel approaches to materials handling like the SA V -concept place new requirements on the AGVS development and the use of simulation as a part of the process. Traditional AGV -system simulation approaches do not fully meet these requirements and the improved functionality of AGVs is not used to its full power. There is a considerflble potential in shortening the AGV -system design-cycle, and thus the manufacturing system design-cycle, and still achieve more accurate solutions well suited for MRS tasks. Recent developments in simulation tools for manufacturing have improved production engineering development and the tools are being adopted more widely in industry. For the development of AGV -systems this has not fully been exploited. Previous research has focused on the conceptual part of the design process and many simulation approaches to AGV -system design lack in validity. In this thesis a methodology is proposed for the structured development of AGV -systems using simulation. Elements of this methodology address the development of novel functionality. The objective of the first research case of this research study was to identify factors for industrial AGV -system simulation. The second research case focuses on simulation in the design of Semi-autonomous vehicles, and the third case evaluates a simulation based design framework. This research study has advanced development by offering a framework for developing testing and evaluating AGV -systems, based on concurrent development using a virtual environment. The ability to exploit unique or novel features of AGVs based on a virtual environment improves the potential of AGV-systems considerably.University of Skovde. European Commission for funding the INCO/COPERNICUS Projec

Unmediated Interaction: Communicating with Computers and Embedded Devices as If They Are Not There

Although computers are smaller and more readily accessible today than they have ever been, I believe that we have barely scratched the surface of what computers can become. When we use computing devices today, we end up spending a lot of our time navigating to particular functions or commands to use devices their way rather than executing those commands immediately. In this dissertation, I explore what I call unmediated interaction, the notion of people using computers as if the computers are not there and as if the people are using their own abilities or powers instead. I argue that facilitating unmediated interaction via personalization, new input modalities, and improved text entry can reduce both input overhead and output overhead, which are the burden of providing inputs to and receiving outputs from the intermediate device, respectively. I introduce three computational methods for reducing input overhead and one for reducing output overhead. First, I show how input data mining can eliminate the need for user inputs altogether. Specifically, I develop a method for mining controller inputs to gain deep insights about a players playing style, their preferences, and the nature of video games that they are playing, all of which can be used to personalize their experience without any explicit input on their part. Next, I introduce gaze locking, a method for sensing eye contact from an image that allows people to interact with computers, devices, and other objects just by looking at them. Third, I introduce computationally optimized keyboard designs for touchscreen manual input that allow people to type on smartphones faster and with far fewer errors than currently possible. Last, I introduce the racing auditory display (RAD), an audio system that makes it possible for people who are blind to play the same types of racing games that sighted players can play, and with a similar speed and sense of control as sighted players. The RAD shows how we can reduce output overhead to provide user interface parity between people with and without disabilities. Together, I hope that these systems open the door to even more efforts in unmediated interaction, with the goal of making computers less like devices that we use and more like abilities or powers that we have

Generalized Model to Enable Zero-shot Imitation Learning for Versatile Robots

The rapid advancement in Deep Learning (DL), especially in Reinforcement Learning (RL) and Imitation Learning (IL), has positioned it as a promising approach for a multitude of autonomous robotic systems. However, the current methodologies are predominantly constrained to singular setups, necessitating substantial data and extensive training periods. Moreover, these methods have exhibited suboptimal performance in tasks requiring long-horizontal maneuvers, such as Radio Frequency Identification (RFID) inventory, where a robot requires thousands of steps to complete. In this thesis, we address the aforementioned challenges by presenting the Cross-modal Reasoning Model (CMRM), a novel zero-shot Imitation Learning policy, to tackle long-horizontal robotic tasks. The RFID inventory task is a typical long-horizontal robotic task that can be formulated as a Partially Observable Markov Decision Process (POMDP); the robot should be able to recall previous actions and reason from current environmental observations to optimize its strategy. To this end, our CMRM has been designed with a two-stream flow structure to extract abstract information concealed in environmental observations and subsequently generate robot actions by reasoning structural and temporal features from historical and current observations. Extensive experiments in a virtual platform and mockup real store are conducted to evaluate the proposed CMRM. Experimental results demonstrate that CMRM is capable of performing RFID inventory tasks in unstructured environments with complex layouts and provides competitive accuracy that surpasses previous methods and manual inventory. To facilitate the training and assessment of CMRM, we constructed a Unity3D-based virtual platform that can be configured into various environments, like an apparel store. This platform is capable of offering photo-realistic objects and precise physical features (gravities, appearance, and more) to provide close to real environments for training and testing robots. Subsequently, the robot, once trained, was deployed in an actual retail environment to perform RFID inventory tasks. This approach effectively bridges the ``reality gap , enabling the robot to perform the RFID inventory task seamlessly in both virtual and real-world settings, thereby demonstrating zero-shot generalization capabilities

Computational Tools and Facilities for the Next-Generation Analysis and Design Environment

This document contains presentations from the joint UVA/NASA Workshop on Computational Tools and Facilities for the Next-Generation Analysis and Design Environment held at the Virginia Consortium of Engineering and Science Universities in Hampton, Virginia on September 17-18, 1996. The presentations focused on the computational tools and facilities for analysis and design of engineering systems, including, real-time simulations, immersive systems, collaborative engineering environment, Web-based tools and interactive media for technical training. Workshop attendees represented NASA, commercial software developers, the aerospace industry, government labs, and academia. The workshop objectives were to assess the level of maturity of a number of computational tools and facilities and their potential for application to the next-generation integrated design environment

Factors Impacting Habitable Volume Requirements: Results from the 2011 Habitable Volume Workshop

This report documents the results of the Habitable Volume Workshop held April 18-21, 2011 in Houston, TX at the Center for Advanced Space Studies-Universities Space Research Association. The workshop was convened by NASA to examine the factors that feed into understanding minimum habitable volume requirements for long duration space missions. While there have been confinement studies and analogs that have provided the basis for the guidance found in current habitability standards, determining the adequacy of the volume for future long duration exploration missions is a more complicated endeavor. It was determined that an improved understanding of the relationship between behavioral and psychosocial stressors, available habitable and net habitable volume, and interior layouts was needed to judge the adequacy of long duration habitat designs. The workshop brought together a multi-disciplinary group of experts from the medical and behavioral sciences, spaceflight, human habitability disciplines and design professionals. These subject matter experts identified the most salient design-related stressors anticipated for a long duration exploration mission. The selected stressors were based on scientific evidence, as well as personal experiences from spaceflight and analogs. They were organized into eight major categories: allocation of space; workspace; general and individual control of environment; sensory deprivation; social monotony; crew composition; physical and medical issues; and contingency readiness. Mitigation strategies for the identified stressors and their subsequent impact to habitat design were identified. Recommendations for future research to address the stressors and mitigating design impacts are presented

Cognitive Mapping for Object Searching in Indoor Scenes

abstract: Visual navigation is a multi-disciplinary field across computer vision, machine learning and robotics. It is of great significance in both research and industrial applications. An intelligent agent with visual navigation ability will be capable of performing the following tasks: actively explore in environments, distinguish and localize a requested target and approach the target following acquired strategies. Despite a variety of advances in mobile robotics, enabling an autonomous with above-mentioned abilities is still a challenging and complex task. However, the solution to the task is very likely to accelerate the landing of assistive robots. Reinforcement learning is a method that trains autonomous robot based on rewarding desired behaviors to help it obtain an action policy that maximizes rewards while the robot interacting with the environment. Through trial and error, an agent learns sophisticated and skillful strategies to handle complex tasks in the environment. Inspired by navigation procedures of human beings that when navigating through environments, humans reason about accessible spaces and geometry of the environment a lot based on first-person view, figure out the destination and then ease over, this work develops a model that maps from pixels to actions and inherently estimate the target as well as the free-space map. The model has three major constituents: (i) a cognitive mapper that maps the topologic free-space map from first-person view images, (ii) a target recognition network that locates a desired object and (iii) an action policy deep reinforcement learning network. Further, a planner model with cascade architecture based on multi-scale semantic top-down occupancy map input is proposed.Dissertation/ThesisMasters Thesis Computer Engineering 201

The design and simulation of traffic networks in virtual environments

For over half a century, researchers from a diverse set of disciplines have been studying the behaviour of traffic flow to better understand the causes of traffic congestion, accidents, and related phenomena. As the global population continues to rise, there is an increasing demand for more efficient and effective transportation infrastructures that are able to accommodate a greater number of civilians without compromising travel times, journey quality, cost, or accessibility. With recent advances in computing technology, transportation infrastructures are now typically developed using design and simulation packages that enable engineers to accurately model large-scale road networks and evaluate their designs through visual simulation. However, as these projects increase in scale and complexity, methodologies to intuitively design more complex and realistic simulations are highly desirable. The need of such technology translates across to the entertainment industry, where traffic simulations are integrated into computer games, television, film, and virtual tourism applications to enhance the realism and believability of the simulated scenario. In this thesis two significant challenges related to the design and simulation of traffic networks for use in virtual environments are presented. The first challenge is the development of intuitive techniques to assist the design and construction of high-fidelity three-dimensional road networks for use in both urban and rural virtual environments. The second challenge considers the implementation of computational models to accurately simulate the behaviour of drivers and pedestrians in transportation networks, in real time. An overview of the literature in the field is presented in this work with novel contributions relating to the challenges defined above