Robotic user interface enabled interactive dialogue with intelligent spaces

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2006.Includes bibliographical references (leaves 81-86).Users can communicate with ubiquitous computing environments by natural means such as voice communication. However, users of the Intelligent Room at MIT CSAIL, a ubiquitous environment, have reported dissatisfaction communicating with the room due to the absence of a focal point and the room's inability to hold a dialogue. To enrich the user's interactive experience, we integrated a Robotic User Interface to the room, and augmented the room's natural language system to enable it to hold dialogues with users. The robotic teddy bear serves two purposes. First, it acts as the focal point of the room which users can address. Second, it enables the room to physically communicate with users by robotic gestures. We also incorporated a book recommendation system to illustrate the room's new ability to converse with users. These enhancements have heightened user experience in communicating with the Intelligent Room, as indicated by our user study.by Rubaiyat Khan.M.Eng

Therapeutic and educational objectives in robot assisted play for children with autism

“This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder." “Copyright IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.” DOI: 10.1109/ROMAN.2009.5326251This article is a methodological paper that describes the therapeutic and educational objectives that were identified during the design process of a robot aimed at robot assisted play. The work described in this paper is part of the IROMEC project (Interactive Robotic Social Mediators as Companions) that recognizes the important role of play in child development and targets children who are prevented from or inhibited in playing. The project investigates the role of an interactive, autonomous robotic toy in therapy and education for children with special needs. This paper specifically addresses the therapeutic and educational objectives related to children with autism. In recent years, robots have already been used to teach basic social interaction skills to children with autism. The added value of the IROMEC robot is that play scenarios have been developed taking children's specific strengths and needs into consideration and covering a wide range of objectives in children's development areas (sensory, communicational and interaction, motor, cognitive and social and emotional). The paper describes children's developmental areas and illustrates how different experiences and interactions with the IROMEC robot are designed to target objectives in these areas

Therapeutic and educational objectives in robot assisted play for children with autism

“This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder." “Copyright IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.” DOI: 10.1109/ROMAN.2009.5326251This article is a methodological paper that describes the therapeutic and educational objectives that were identified during the design process of a robot aimed at robot assisted play. The work described in this paper is part of the IROMEC project (Interactive Robotic Social Mediators as Companions) that recognizes the important role of play in child development and targets children who are prevented from or inhibited in playing. The project investigates the role of an interactive, autonomous robotic toy in therapy and education for children with special needs. This paper specifically addresses the therapeutic and educational objectives related to children with autism. In recent years, robots have already been used to teach basic social interaction skills to children with autism. The added value of the IROMEC robot is that play scenarios have been developed taking children's specific strengths and needs into consideration and covering a wide range of objectives in children's development areas (sensory, communicational and interaction, motor, cognitive and social and emotional). The paper describes children's developmental areas and illustrates how different experiences and interactions with the IROMEC robot are designed to target objectives in these areas.Final Published versio

Exploring Robot Teleoperation in Virtual Reality

This thesis presents research on VR-based robot teleoperation with a focus on remote environment visualisation in virtual reality, the effects of remote environment reconstruction scale in virtual reality on the human-operator's ability to control the robot and human-operator's visual attention patterns when teleoperating a robot from virtual reality. A VR-based robot teleoperation framework was developed, it is compatible with various robotic systems and cameras, allowing for teleoperation and supervised control with any ROS-compatible robot and visualisation of the environment through any ROS-compatible RGB and RGBD cameras. The framework includes mapping, segmentation, tactile exploration, and non-physically demanding VR interface navigation and controls through any Unity-compatible VR headset and controllers or haptic devices. Point clouds are a common way to visualise remote environments in 3D, but they often have distortions and occlusions, making it difficult to accurately represent objects' textures. This can lead to poor decision-making during teleoperation if objects are inaccurately represented in the VR reconstruction. A study using an end-effector-mounted RGBD camera with OctoMap mapping of the remote environment was conducted to explore the remote environment with fewer point cloud distortions and occlusions while using a relatively small bandwidth. Additionally, a tactile exploration study proposed a novel method for visually presenting information about objects' materials in the VR interface, to improve the operator's decision-making and address the challenges of point cloud visualisation. Two studies have been conducted to understand the effect of virtual world dynamic scaling on teleoperation flow. The first study investigated the use of rate mode control with constant and variable mapping of the operator's joystick position to the speed (rate) of the robot's end-effector, depending on the virtual world scale. The results showed that variable mapping allowed participants to teleoperate the robot more effectively but at the cost of increased perceived workload. The second study compared how operators used a virtual world scale in supervised control, comparing the virtual world scale of participants at the beginning and end of a 3-day experiment. The results showed that as operators got better at the task they as a group used a different virtual world scale, and participants' prior video gaming experience also affected the virtual world scale chosen by operators. Similarly, the human-operator's visual attention study has investigated how their visual attention changes as they become better at teleoperating a robot using the framework. The results revealed the most important objects in the VR reconstructed remote environment as indicated by operators' visual attention patterns as well as their visual priorities shifts as they got better at teleoperating the robot. The study also demonstrated that operators’ prior video gaming experience affects their ability to teleoperate the robot and their visual attention behaviours

Combined 3D printing and part assembly manufacturing

Despite its irrefutable potential to change the way goods are produced, 3D printing technologies are, nowadays, employed to an insignificant amount of manufacturing processes in the production chain. While additive manufacturing, as the technology is also known, can be used for building objects of any shapes in a wide range of materials, the resulting products usually require costly post-processing steps for the incorporation of nonprintable parts that provide the object with useful functionalities. Such technologies could play a more important role in the manufacturing sector if they were enhanced so that these post-processing steps would be addressed during the building process and the resulting object would already be equipped with its final set of features. Aiming to add more functionalities to the parts produced by 3D printing techniques, this study introduces improvements to be applied to additive manufacturing machines. The technology introduced and demonstrated in this research, named Combined 3D Printing and Part Assembly (CPPA), merges the versatility of 3D printing with techniques to assemble external parts to the printed object in building time, allowing the manufacturing of products with functionalities beyond the ones provided by classic additive methods. While traditional 3D printing technologies can only create objects in a limited variety of materials, CPPA machines are capable of merging any sort of external parts to 3D printed structures.Apesar de seu irrefutável potencial de mudar o modo como bens são produzidos, a tecnologia de impressão 3D são empregadas em uma insignificante parcela dos processos de manufatura na cadeia produtiva. Ao mesmo tempo que a manufatura aditiva, como a tecnologia também é conhecida, é capaz de gerar objetos de qualquer formato em uma imensa gama de materiais, os produtos resultantes frequentemente necessitam de etapas de pós-processamento altamente custosas para a incorporação de partes não imprimíveis que proporcionam funcionalidades úteis ao objeto. Tais tecnologias poderiam exercer um papel mais importante no setor de manufatura se passassem por melhorias que permitissem que essas etapas de pós-processamento fossem executadas durante o próprio processo de impressão, de forma que o o objeto resultante já estivesse equipado com o seu conjunto de funcionalidades final. Visando adicionar mais funcionalidades aos objetos produzidos com técnicas de impressão 3D, esse estudo apresenta aperfeiçoamentos a serem aplicados em equipamentos de manufatura aditiva. A tecnologia denominada Combined 3D Printing and Part Assembly (CPPA) introduzida e demonstrada nessa pesquisa tem por objetivo combinar a versatilidade da impressão 3D com a anexação de peças externas ao objeto em tempo de impressão, de modo a criar produtos com funcionalidades além das fornecidas por métodos aditivos clássicos. Enquanto tecnologias tradicionais de impressão 3D criam objetos com uma variedade muito limitada de materiais, máquinas CPPA são capazes de incorporar peças externas de qualquer tipo a estruturas impressas em 3D

A contribution to the incorporation of sociability and creativity skills to computers and robots

This dissertation contains the research and work completed by the PhD candidate on the incorporation of sociability and creativity skills to computers and robots. Both skills can be directly related with empathy, which is the ability to understand and share the feelings of another. In this form, this research can be contextualized in the framework of recent developments towards the achievement of empathy machines. The first challenge at hands refers to designing pioneering techniques based on the use of social robots to improve user experience interacting with them. In particular, research focus is on eliminating or minimizing pain and anxiety as well as loneliness and stress of long-term hospitalized child patients. This challenge is approached by developing a cloud-based robotics architecture to effectively develop complex tasks related to hospitalized children assistance. More specifically, a multiagent learning system is introduced based on a combination of machine learning and cloud computing using low-cost robots (Innvo labs's Pleo rb). Moreover, a wireless communication system is also developed for the Pleo robot in order to help the health professional who conducts therapy with the child, monitoring, understanding, and controlling Pleo behavior at any moment. As a second challenge, a new formulation of the concept of creativity is proposed in order to empower computers with. Based on previous well established theories from Boden and Wiggins, this thesis redefines the formal mechanism of exploratory and transformational creativity in a way which facilitates the computational implementation of these mechanisms in Creativity Support Systems. The proposed formalization is applied and validated on two real cases: the first, about chocolate designing, in which a novel and flavorful combination of chocolate and fruit is generated. The second case is about the composition of a single voice tune of reel using ABC notation.Postprint (published version