Affordance-based task communication methods for astronaut-robot cooperation

The problem with current human-robot task communication is that robots cannot understand complex human speech utterances, while humans cannot efficiently use the fixed task request utterances required by robots. Nonetheless, future planetary exploration missions are expected to require astronauts on extra-vehicular activities to communicate task requests to robot assistants with speech- and gesture-type user interfaces that can be easily embedded in their space suits. The solution proposed in this thesis is indirect task communication based on the human-like ability to utilise object-action relationships in task communication. Conventional task communication methods, in which all task parameters need to be communicated explicitly, are evaluated against task communication methods where affordances, i.e. action possibilities, are used to complete task communication. These so-called affordance-based task communication methods are evaluated by means of four user experiments: two performed with a fully autonomous centauroid robot in a planetary exploration work context and two with a simulated robot in a lander assembly work context. The first two experiments are performed in unambiguous work environments, where each object is associated with only one action and vice versa, while the last two experiments are performed in ambiguous work environments, where each object and action is normally associated with several actions and objects, respectively. The user experiments show that affordance-based task communication methods can be used to decrease both the human workload and task communication times in a planetary exploration work context. Furthermore, affordance-based task communication methods are found to be preferred over conventional task communication methods. The affordance-based task communication methods derived can be applied to facilitate any human-robot task communication that includes a priori known or recurring task sequences. In this thesis, the feasibility of the approach was demonstrated for frame-based dialogue managers, which are widely used in robotics

An Object Template Approach to Manipulation for Semi-autonomous Avatar Robots

Nowadays, the first steps towards the use of mobile robots to perform manipulation tasks in remote environments have been made possible. This opens new possibilities for research and development, since robots can help humans to perform tasks in many scenarios. A remote robot can be used as avatar in applications such as for medical or industrial use, in rescue and disaster recovery tasks which might be hazardous environments for human beings to enter, as well as for more distant scenarios like planetary explorations. Among the most typical applications in recent years, research towards the deployment of robots to mitigate disaster scenarios has been of great interest in the robotics field. Disaster scenarios present challenges that need to be tackled. Their unstructured nature makes them difficult to predict and even though some assumptions can be made for human-designed scenarios, there is no certainty on the expected conditions. Communications with a robot inside these scenarios might also be challenged; wired communications limit reachability and wireless communications are limited by bandwidth. Despite the great progress in the robotics research field, these difficulties have prevented the current autonomous robotic approaches to perform efficiently in unstructured remote scenarios. On one side, acquiring physical and abstract information from unknown objects in a full autonomous way in uncontrolled environmental conditions is still an unsolved problem. Several challenges have to be overcome such as object recognition, grasp planning, manipulation, and mission planning among others. On the other side, purely teleoperated robots require a reliable communication link robust to reachability, bandwidth, and latency which can provide all the necessary feedback that a human operator needs in order to achieve sufficiently good situational awareness, e.g., worldmodel, robot state, forces, and torques exerted. Processing this amount of information plus the necessary training to perform joint motions with the robot represent a high mental workload for the operator which results in very low execution times. Additionally, a pure teleoperated approach is error-prone given that the success in a manipulation task strongly depends on the ability and expertise of the human operating the robot. Both, autonomous and teleoperated robotic approaches have pros and cons, for this reason a middle ground approach has emerged. In an approach where a human supervises a semi-autonomous remote robot, strengths from both, full autonomous and purely teleoperated approaches can be combined while at the same time their weaknesses can be tackled. A remote manipulation task can be divided into sub-tasks such as planning, perception, action, and evaluation. A proper distribution of these sub-tasks between the human operator and the remote robot can increase the efficiency and potential of success in a manipulation task. On the one hand, a human operator can trivially plan a task (planning), identify objects in the sensor data acquired by the robot (perception), and verify the completion of a task (evaluation). On the other hand, it is challenging to remotely control in joint space a robotic system like a humanoid robot that can easily have over 25 degrees of freedom (DOF). For this reason, in this approach the complex sub-tasks such as motion planning, motion execution, and obstacle avoidance (action) are performed autonomously by the remote robot. With this distribution of tasks, the challenge of converting the operator intent into a robot action arises. This thesis investigates concepts of how to efficiently provide a remote robot with the operator intent in a flexible means of interaction. While current approaches focus on an object-grasp-centered means of interaction, this thesis aims at providing physical and abstract properties of the objects of interest. With this information, the robot can perform autonomous subtasks like locomotion through the environment, grasping objects, and manipulating them at an affordance-level avoiding collisions with the environment in order to efficiently accomplish the manipulation task needed. For this purpose, the concept of Object Template (OT) has been developed in this thesis. An OT is a virtual representation of an object of interest that contains information that a remote robot can use to manipulate such object or other similar objects. The object template concept presented here goes beyond state-of-the-art related concepts by extending the robot capabilities to use affordance information of the object. This concept includes physical information (mass, center of mass, inertia tensor) as well as abstract information (potential grasps, affordances, and usabilities). Because humans are very good at analysing a situation, planning new ways of how to solve a task, even using objects for different purposes, it is important to allow communicating the planning and perception performed by the operator such that the robot can execute the action based on the information contained in the OT. This leverages human intelligence with robot capabilities. For example, as an implementation in a 3D environment, an OT can be visualized as a 3D geometry mesh that simulates an object of interest. A human operator can manipulate the OT and move it so that it overlaps with the visualized sensor data of the real object. Information of the object template type and its pose can be compressed and sent using low bandwidth communication. Then, the remote robot can use the information of the OT to approach, grasp, and manipulate the real object. The use of remote humanoid robots as avatars is expected to be intuitive to operators (or potential human response forces) since the kinematic chains and degrees of freedom are similar to humans. This allows operators to visualize themselves in the remote environment and think how to solve a task, however, task requirements such as special tools might not be found. For this reason, a flexible means of interaction that can account for allowing improvisation from the operator is also needed. In this approach, improvisation is described as "a change of a plan on how to achieve a certain task, depending on the current situation". A human operator can then improvise by adapting the affordances of known objects into new unknown objects. For example, by utilizing the affordances defined in an OT on a new object that has similar physical properties or which manipulation skills belong to the same class. The experimental results presented in this thesis validate the proposed approach by demonstrating the successful achievement of several manipulation tasks using object templates. Systematic laboratory experimentation has been performed to evaluate the individual aspects of this approach. The performance of the approach has been tested in three different humanoid robotic systems (one of these robots belongs to another research laboratory). These three robotic platforms also participated in the renowned international competition DARPA Robotics Challenge (DRC) which between 2012 and 2015 was considered the most ambitious and challenging robotic competition

Design Space for Space Design: Humanly {S:pace} Constructs Across Perceptual Boundaries

In this PhD research by thesis, the author documents his journey that explores modes of operations beyond those predominantly applied at NASA. Specifically, he is looking at designerly and artistic modes of operation, with a research goal to show demonstrable value to enhance NASA’s capability to innovate. This exploration is built on cybernetic perspectives and goal-seeking focused on human centered design within NASA’s space exploration paradigm. The author uses a performative approach through real world examples to highlight and substantiate the benefits of novel perspectives, conversations, and boundary objects, which shows their demonstrable value to NASA. The significance of the research findings is discussed in relations to the state of practice, which is derived from interviews with practitioners across NASA’s organizational hierarchy, combined with personal experiences, and independent research on the topics. The two primary application examples examine strategic level organizational conversations in support of strategic decision-making, and a human centered approach to space habitats that utilizes conversations and boundary objects aimed towards higher-level needs of the astronauts. Secondary examples, as added material, explore designing the design environments through human centered conversations with stakeholders, storytelling, multi-nodal and multimodal conversations, designerly modes of operation in engineering-focused environments, and explore the potential benefits of a design education program to change the organizational culture on the long term. These examples are grounded and substantiated using specifically created boundary objects, which are used as communication tools across multiple disciplines. This research is timely, because expanding humanity into space is an ongoing and inevitable step in our quest to explore our world. Yet space exploration is costly, and the awaiting environment challenges us, the human explorers, with extreme cold, heat, vacuum and radiation—among other conditions—unlike anything encountered on Earth. As a consequence, today’s space exploration, both robotic- and human-exploration driven, is dominated by objects and artifacts which are mostly conceived, designed and built through technological and engineering approaches, to support basic physiological, psychological, and safety needs. NASA’s activities, products, and processes are controlled by rigid procedural requirements, and are highly dependent on government funding. Since the Apollo era, the annual budget decreased by nine fold and remained virtually flat. Resource constraints, funding uncertainty, and changes in the organizational culture gradually led to innovation barriers, and formed a temporally and spatially coupled cyclical wicked problem for NASA. Yet, the aging workforce, still remembering the golden age of space exploration, is hoping and planning for large “fire and smoke” type missions, which puts NASA on an unsustainable path, while perpetuated by technology and management focus to overcome obstacles. Finding new directions may require a second-order cybernetic transformational change, starting with a changed paradigm, which in turn will impact the Agency’s mission and culture, and influence the core processes. In this research the author makes a case to broaden NASA’s worldview today, which is dominated by science, engineering, technology, project and resource management considerations. This can be achieved through novel perspectives gained from cybernetics, and other modes of operation through human centered design and art. While the proposed performative approach is applied to NASA, it is not bounded by it. These perspectives and modes of operation can be applied to any other field, discipline or hierarchical structure within scientific, technological, and social developments. Cybernetic mapping of any environment can provide insights to the connections and the potential for interactions between the various actors within. Understanding the complexities, non-linearity, and competing and often misaligned influences is important to set goals for the system and navigate towards preferable outcomes. Controlling and regulating the variety of these dynamic and responsive systems, in line with the set out goals and objectives, also require considerations and guidance, where cybernetic mapping, conversations and novel shared languages between the actors (in the form of commonly agreed understanding of the meaning), and human center design may play a role. When people are involved in these circular interactions and conversations, human centeredness can lead to transformative psychological impact on a personal level, and strategic advantages at an organizational level

Annals of Scientific Society for Assembly, Handling and Industrial Robotics

This Open Access proceedings present a good overview of the current research landscape of industrial robots. The objective of MHI Colloquium is a successful networking at academic and management level. Thereby the colloquium is focussing on a high level academic exchange to distribute the obtained research results, determine synergetic effects and trends, connect the actors personally and in conclusion strengthen the research field as well as the MHI community. Additionally there is the possibility to become acquainted with the organizing institute. Primary audience are members of the scientific association for assembly, handling and industrial robots (WG MHI)

Human factors in instructional augmented reality for intravehicular spaceflight activities and How gravity influences the setup of interfaces operated by direct object selection

In human spaceflight, advanced user interfaces are becoming an interesting mean to facilitate human-machine interaction, enhancing and guaranteeing the sequences of intravehicular space operations. The efforts made to ease such operations have shown strong interests in novel human-computer interaction like Augmented Reality (AR). The work presented in this thesis is directed towards a user-driven design for AR-assisted space operations, iteratively solving issues arisen from the problem space, which also includes the consideration of the effect of altered gravity on handling such interfaces.Auch in der bemannten Raumfahrt steigt das Interesse an neuartigen Benutzerschnittstellen, um nicht nur die Mensch-Maschine-Interaktion effektiver zu gestalten, sondern auch um einen korrekten Arbeitsablauf sicherzustellen. In der Vergangenheit wurden wiederholt Anstrengungen unternommen, Innenbordarbeiten mit Hilfe von Augmented Reality (AR) zu erleichtern. Diese Arbeit konzentriert sich auf einen nutzerorientierten AR-Ansatz, welcher zum Ziel hat, die Probleme schrittweise in einem iterativen Designprozess zu lösen. Dies erfordert auch die Berücksichtigung veränderter Schwerkraftbedingungen

An Enactivist Model of Improvisational Dance

An Enactivist Model of Improvisational Danc

Enhancing Free-text Interactions in a Communication Skills Learning Environment

Learning environments frequently use gamification to enhance user interactions.Virtual characters with whom players engage in simulated conversations often employ prescripted dialogues; however, free user inputs enable deeper immersion and higher-order cognition. In our learning environment, experts developed a scripted scenario as a sequence of potential actions, and we explore possibilities for enhancing interactions by enabling users to type free inputs that are matched to the pre-scripted statements using Natural Language Processing techniques. In this paper, we introduce a clustering mechanism that provides recommendations for fine-tuning the pre-scripted answers in order to better match user inputs