Data-driven learning for robot physical intelligence

The physical intelligence, which emphasizes physical capabilities such as dexterous manipulation and dynamic mobility, is essential for robots to physically coexist with humans. Much research on robot physical intelligence has achieved success on hyper robot motor capabilities, but mostly through heavily case-specific engineering. Meanwhile, in terms of robot acquiring skills in a ubiquitous manner, robot learning from human demonstration (LfD) has achieved great progress, but still has limitations handling dynamic skills and compound actions. In this dissertation, a composite learning scheme which goes beyond LfD and integrates robot learning from human definition, demonstration, and evaluation is proposed. This method tackles advanced motor skills that require dynamic time-critical maneuver, complex contact control, and handling partly soft partly rigid objects. Besides, the power of crowdsourcing is brought to tackle case-specific engineering problem in the robot physical intelligence. Crowdsourcing has demonstrated great potential in recent development of artificial intelligence. Constant learning from a large group of human mentors breaks the limit of learning from one or a few mentors in individual cases, and has achieved success in image recognition, translation, and many other cyber applications. A robot learning scheme that allows a robot to synthesize new physical skills using knowledge acquired from crowdsourced human mentors is proposed. The work is expected to provide a long-term and big-scale measure to produce advanced robot physical intelligence

Pseudo-haptics survey: Human-computer interaction in extended reality & teleoperation

Pseudo-haptic techniques are becoming increasingly popular in human-computer interaction. They replicate haptic sensations by leveraging primarily visual feedback rather than mechanical actuators. These techniques bridge the gap between the real and virtual worlds by exploring the brain’s ability to integrate visual and haptic information. One of the many advantages of pseudo-haptic techniques is that they are cost-effective, portable, and flexible. They eliminate the need for direct attachment of haptic devices to the body, which can be heavy and large and require a lot of power and maintenance. Recent research has focused on applying these techniques to extended reality and mid-air interactions. To better understand the potential of pseudo-haptic techniques, the authors developed a novel taxonomy encompassing tactile feedback, kinesthetic feedback, and combined categories in multimodal approaches, ground not covered by previous surveys. This survey highlights multimodal strategies and potential avenues for future studies, particularly regarding integrating these techniques into extended reality and collaborative virtual environments.info:eu-repo/semantics/publishedVersio

Reusing a robot's behavioral mechanisms to model and manipulate human mental states

Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2010.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Cataloged from student submitted PDF version of thesis.Includes bibliographical references (p. 125-129).In a task domain characterized by physical actions and where information has value, competing teams gain advantage by spying on and deceiving an opposing team while cooperating teammates can help the team by secretly communicating new information. For a robot to thrive in this environment it must be able to perform actions in a manner to deceive opposing agents as well as to be able to secretly communicate with friendly agents. It must further be able to extract information from observing the actions of other agents. The goal of this research is to expand on current human robot interaction by creating a robot that can operate in the above scenario. To enable these behaviors, an architecture is created which provides the robot with mechanisms to work with hidden human mental states. The robot attempts to infer these hidden states from observable factors and use them to better understand and predict behavior. It also takes steps to alter them in order to change the future behavior of the other agent. It utilizes the knowledge that the human is performing analogous inferences about the robot's own internal states to predict the effect of its actions on the human's knowledge and perceptions of the robot. The research focuses on the implicit communication that is made possible by two embodied agents interacting in a shared space through nonverbal interaction. While the processes used by a robot differ significantly from the cognitive mechanisms employed by humans, each face the similar challenge of completing the loop from sensing to acting. This architecture employs a self-as-simulator strategy, reusing the robot's behavioral mechanisms to model aspects of the human's mental states. This reuse allows the robot to model human actions and the mental states behind them using the grammar of its own representations and actions.by Jesse Vail Gray.Ph.D

I am here - are you there? Sense of presence and implications for virtual world design

We use the language of presence and place when we interact online: in our instant text messaging windows we often post: Are you there? Research indicates the importance of the sense of presence for computer-supported collaborative virtual learning. To realize the potential of virtual worlds such as Second Life, which may have advantages over conventional text-based environments, we need an understanding of design and the emergence of the sense of presence. A construct was created for the sense of presence, as a collaborative, action-based process (Spagnolli, Varotto, & Mantovani, 2003) with four dimensions (sense of place, social presence, individual agency, and mediated collaborative actions). Nine design principles were mapped against the four dimensions. The guiding question for the study\u27s exploration of the sense of presence was: In the virtual world Second Life, what is the effect on the sense of presence in collaborative learning spaces designed according to the sense of presence construct proposed, using two of the nine design principles, wayfinding and annotation? Another question of interest was: What are the relationships, if any, among the four dimensions of presence? The research utilized both quantitative and qualitative measures. Twenty learners recruited from the Graduate School of Education and Psychology at Pepperdine University carried out three assigned collaborative activities in Second Life under design conditions foregrounding each of the two design conditions, and a combination of the two. Analyses from surveys, Second Life interactions, interviews and a focus group were conducted to investigate how various designed learning environments based in the virtual world contributed to the sense of presence, and to learners\u27 ability to carry out collaborative learning. The major research findings were: (a) the construct appears robust, and future research in its application to other virtual worlds may be fruitful; (b) the experience of wayfinding (finding a path through a virtual space) resulted overall in an observed pattern of a slightly stronger sense of place; (c) the experience of annotation (building) resulted overall in an observed pattern of a slightly stronger sense of agency; and (d) there is a positive association between sense of place and sense of agency

Internet of robotic things : converging sensing/actuating, hypoconnectivity, artificial intelligence and IoT Platforms

The Internet of Things (IoT) concept is evolving rapidly and influencing newdevelopments in various application domains, such as the Internet of MobileThings (IoMT), Autonomous Internet of Things (A-IoT), Autonomous Systemof Things (ASoT), Internet of Autonomous Things (IoAT), Internetof Things Clouds (IoT-C) and the Internet of Robotic Things (IoRT) etc.that are progressing/advancing by using IoT technology. The IoT influencerepresents new development and deployment challenges in different areassuch as seamless platform integration, context based cognitive network integration,new mobile sensor/actuator network paradigms, things identification(addressing, naming in IoT) and dynamic things discoverability and manyothers. The IoRT represents new convergence challenges and their need to be addressed, in one side the programmability and the communication ofmultiple heterogeneous mobile/autonomous/robotic things for cooperating,their coordination, configuration, exchange of information, security, safetyand protection. Developments in IoT heterogeneous parallel processing/communication and dynamic systems based on parallelism and concurrencyrequire new ideas for integrating the intelligent “devices”, collaborativerobots (COBOTS), into IoT applications. Dynamic maintainability, selfhealing,self-repair of resources, changing resource state, (re-) configurationand context based IoT systems for service implementation and integrationwith IoT network service composition are of paramount importance whennew “cognitive devices” are becoming active participants in IoT applications.This chapter aims to be an overview of the IoRT concept, technologies,architectures and applications and to provide a comprehensive coverage offuture challenges, developments and applications