Action Representations in Robotics: A Taxonomy and Systematic Classification

Understanding and defining the meaning of "action" is substantial for robotics research. This becomes utterly evident when aiming at equipping autonomous robots with robust manipulation skills for action execution. Unfortunately, to this day we still lack both a clear understanding of the concept of an action and a set of established criteria that ultimately characterize an action. In this survey we thus first review existing ideas and theories on the notion and meaning of action. Subsequently we discuss the role of action in robotics and attempt to give a seminal definition of action in accordance with its use in robotics research. Given this definition we then introduce a taxonomy for categorizing action representations in robotics along various dimensions. Finally, we provide a systematic literature survey on action representations in robotics where we categorize relevant literature along our taxonomy. After discussing the current state of the art we conclude with an outlook towards promising research directions.Comment: 36 pages, 4 figures, 7 tables, submitted to the International Journal of Robotics Research (IJRR

Apprentissage permanent par feedback endogène, application à un système robotique

Les applications robotiques sont liées à l'environnement sociotechnique dynamique dans lequel elles sont intégrées. Dans ce contexte, l'auto-adaptation est une préoccupation centrale et la conception d'applications intelligentes dans de tels environnements nécessite de les considérer comme des systèmes complexes. Le domaine de la robotique est très vaste. L'accent est mis sur les systèmes qui s'adaptent aux contraintes de leur environnement et non sur la mécanique ou le traitement du signal. À la lumière de ce contexte, l'objectif de cette thèse est la conception d'un mécanisme d'apprentissage capable d'apprendre de manière continue en utilisant des feedbacks endogènes (i.e. des interactions internes) dans des environnements sociotechniques dynamiques. Ce mécanisme d'apprentissage doit aussi vérifier plusieurs propriétés qui sont essentielles dans ce contexte comme : l'agnosticité, l'apprentissage tout au long de la vie, l'apprentissage en ligne, l'auto-observation, la généralisation des connaissances, le passage à l'échelle, la tolérance au volume de données et l'explicabilité. Les principales contributions consistent en la construction de l'apprentissage endogène par contextes et la conception du mécanisme d'apprentissage ELLSA pour Endogenous Lifelong Learner by Self-Adaptation. Le mécanisme d'apprentissage proposé est basé sur les systèmes multi-agents adaptatifs combinés à l'apprentissage endogène par contextes. La création de l'apprentissage endogène par contextes est motivée par la caractérisation d'imprécisions d'apprentissage qui sont détectées par des négociations locales entre agents. L'apprentissage endogène par contextes comprends aussi un mécanisme de génération de données artificielles pour améliorer les modèles d'apprentissage tout en réduisant la quantité nécessaire de données d'apprentissage. Dans un contexte d'apprentissage tout au long de la vie, ELLSA permet une mise à jour dynamique des modèles d'apprentissage. Il introduit des stratégies d'apprentissage actif et d'auto-apprentissage pour résoudre les imprécisions d'apprentissage. L'utilisation de ces stratégies dépend de la disponibilité des données d'apprentissage. Afin d'évaluer ses contributions, ce mécanisme est appliqué à l'apprentissage de fonctions mathématiques et à un problème réel dans le domaine de la robotique : le problème de la cinématique inverse. Le scénario d'application est l'apprentissage du contrôle de bras robotiques multi-articulés. Les expériences menées montrent que l'apprentissage endogène par contextes permet d'améliorer les performances d'apprentissage grâce à des mécanismes internes. Elles mettent aussi en évidence des propriétés du système selon les objectifs de la thèse : feedback endogènes, agnosticité, apprentissage tout au long de la vie, apprentissage en ligne, auto-observation, généralisation, passage à l'échelle, tolérance au volume de données et explicabilité.Robotic applications are linked to the dynamic sociotechnical environment in which they are embedded. In this scope, self-adaptation is a central concern and the design of intelligent applications in such environments requires to consider them as complex systems. The field of robotics is very broad. The focus is made on systems that adapt to the constraints of their environment and not on mechanics or signal processing. In light of this context, the objective of this thesis is the design of a learning mechanism capable of continuous learning using endogenous feedback (i.e. internal interactions) in dynamic sociotechnical environments. This learning mechanism must also verify several properties that are essential in this context such as: agnosticity, lifelong learning, online learning, self-observation, knowledge generalization, scalability, data volume tolerance and explainability. The main contributions consist of the construction of Endogenous Context Learning and the design of the learning mechanism ELLSA for Endogenous Lifelong Learner by Self-Adaptation. The proposed learning mechanism is based on Adaptive Multi-Agent Systems combined with Context Learning. The creation of Endogenous Context Learning is motivated by the characterization of learning inaccuracies that are detected by local negotiations between agents. Endogenous Context Learning also includes an artificial data generation mechanism to improve learning models while reducing the amount of the required learning data. In a Lifelong Learning setting, ELLSA enables dynamic updating of learning models. It introduces Active Learning and Self-Learning strategies to resolve learning inaccuracies. The use of these strategies depends on the availability of learning data. In order to evaluate its contributions, this mechanism is applied to the learning of mathematical functions and to a real problem in the field of robotics: the Inverse Kinematics problem. The application scenario is the learning of the control of multi-jointed robotic arms. The conducted experiments show that Endogenous Context Learning enables to improve the learning performances thanks to internal mechanisms. They also highlight the properties of the system according to the objectives of the thesis: endogenous feedback, agnosticity, lifelong learning, online learning, self-observation, knowledge generalization, scalability, data volume tolerance and explainability

On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models

This paper addresses the general problem of reinforcement learning (RL) in partially observable environments. In 2013, our large RL recurrent neural networks (RNNs) learned from scratch to drive simulated cars from high-dimensional video input. However, real brains are more powerful in many ways. In particular, they learn a predictive model of their initially unknown environment, and somehow use it for abstract (e.g., hierarchical) planning and reasoning. Guided by algorithmic information theory, we describe RNN-based AIs (RNNAIs) designed to do the same. Such an RNNAI can be trained on never-ending sequences of tasks, some of them provided by the user, others invented by the RNNAI itself in a curious, playful fashion, to improve its RNN-based world model. Unlike our previous model-building RNN-based RL machines dating back to 1990, the RNNAI learns to actively query its model for abstract reasoning and planning and decision making, essentially "learning to think." The basic ideas of this report can be applied to many other cases where one RNN-like system exploits the algorithmic information content of another. They are taken from a grant proposal submitted in Fall 2014, and also explain concepts such as "mirror neurons." Experimental results will be described in separate papers.Comment: 36 pages, 1 figure. arXiv admin note: substantial text overlap with arXiv:1404.782

Applying science of learning in education: Infusing psychological science into the curriculum

The field of specialization known as the science of learning is not, in fact, one field. Science of learning is a term that serves as an umbrella for many lines of research, theory, and application. A term with an even wider reach is Learning Sciences (Sawyer, 2006). The present book represents a sliver, albeit a substantial one, of the scholarship on the science of learning and its application in educational settings (Science of Instruction, Mayer 2011). Although much, but not all, of what is presented in this book is focused on learning in college and university settings, teachers of all academic levels may find the recommendations made by chapter authors of service. The overarching theme of this book is on the interplay between the science of learning, the science of instruction, and the science of assessment (Mayer, 2011). The science of learning is a systematic and empirical approach to understanding how people learn. More formally, Mayer (2011) defined the science of learning as the “scientific study of how people learn” (p. 3). The science of instruction (Mayer 2011), informed in part by the science of learning, is also on display throughout the book. Mayer defined the science of instruction as the “scientific study of how to help people learn” (p. 3). Finally, the assessment of student learning (e.g., learning, remembering, transferring knowledge) during and after instruction helps us determine the effectiveness of our instructional methods. Mayer defined the science of assessment as the “scientific study of how to determine what people know” (p.3). Most of the research and applications presented in this book are completed within a science of learning framework. Researchers first conducted research to understand how people learn in certain controlled contexts (i.e., in the laboratory) and then they, or others, began to consider how these understandings could be applied in educational settings. Work on the cognitive load theory of learning, which is discussed in depth in several chapters of this book (e.g., Chew; Lee and Kalyuga; Mayer; Renkl), provides an excellent example that documents how science of learning has led to valuable work on the science of instruction. Most of the work described in this book is based on theory and research in cognitive psychology. We might have selected other topics (and, thus, other authors) that have their research base in behavior analysis, computational modeling and computer science, neuroscience, etc. We made the selections we did because the work of our authors ties together nicely and seemed to us to have direct applicability in academic settings

Multimodal Data Analysis of Dyadic Interactions for an Automated Feedback System Supporting Parent Implementation of Pivotal Response Treatment

abstract: Parents fulfill a pivotal role in early childhood development of social and communication skills. In children with autism, the development of these skills can be delayed. Applied behavioral analysis (ABA) techniques have been created to aid in skill acquisition. Among these, pivotal response treatment (PRT) has been empirically shown to foster improvements. Research into PRT implementation has also shown that parents can be trained to be effective interventionists for their children. The current difficulty in PRT training is how to disseminate training to parents who need it, and how to support and motivate practitioners after training. Evaluation of the parents’ fidelity to implementation is often undertaken using video probes that depict the dyadic interaction occurring between the parent and the child during PRT sessions. These videos are time consuming for clinicians to process, and often result in only minimal feedback for the parents. Current trends in technology could be utilized to alleviate the manual cost of extracting data from the videos, affording greater opportunities for providing clinician created feedback as well as automated assessments. The naturalistic context of the video probes along with the dependence on ubiquitous recording devices creates a difficult scenario for classification tasks. The domain of the PRT video probes can be expected to have high levels of both aleatory and epistemic uncertainty. Addressing these challenges requires examination of the multimodal data along with implementation and evaluation of classification algorithms. This is explored through the use of a new dataset of PRT videos. The relationship between the parent and the clinician is important. The clinician can provide support and help build self-efficacy in addition to providing knowledge and modeling of treatment procedures. Facilitating this relationship along with automated feedback not only provides the opportunity to present expert feedback to the parent, but also allows the clinician to aid in personalizing the classification models. By utilizing a human-in-the-loop framework, clinicians can aid in addressing the uncertainty in the classification models by providing additional labeled samples. This will allow the system to improve classification and provides a person-centered approach to extracting multimodal data from PRT video probes.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Social Feedback: Social Learning from Interaction History to Support Information Seeking on the Web

Information seeking on the Web has become a central part of many daily activities. Even though information seeking is extremely common, there are many times when these tasks are unsuccessful, because the information found is less than ideal or the task could have been completed more efficiently. In unsuccessful information-seeking tasks, there are often other people who have knowledge or experience that could help improve task success. However, information seekers do not typically look for help from others, because tasks can often be completed alone (even if inefficiently). One of the problems is that web tools provide people with few opportunities to learn from one another’s experiences in ways that would allow them to improve their success. This dissertation presents the idea of social feedback. Social feedback is based on the theory of social learning, which describes how people learn from observing others. In social feedback, observational learning is enabled through the mechanism of interaction history – the traces of activity people create as they interact with the Web. Social feedback systems collect and display interaction history to allow information seekers to learn how to complete their tasks more successfully by observing how other people have behaved in similar situations. The dissertation outlines the design of two social-feedback systems, and describes two studies that demonstrate the real world applicability and feasibility of the idea. The first system supports global learning, by allowing people to learn new search skills and techniques that improve information seeking success in many different tasks. The second system supports local learning, in which people learn how to accomplish specific tasks more effectively and more efficiently. Two further studies are conducted to explore potential real-world challenges to the successful deployment of social feedback systems, such as the privacy concerns associated with the collection and sharing of interaction history. These studies show that social feedback systems can be deployed successfully for supporting real world information seeking tasks. Overall, this research shows that social feedback is a valuable new idea for the social use of information systems, an idea that allows people to learn from one another’s experiences and improve their success in many common real-world tasks

Perceptions of Problem-Based Learning and Attitudes Towards its Adoption Among K-12 Teachers in Seventh-day Adventist Schools in Florida

Problem One of the founders of Seventh-day Adventist education, Ellen G. White, advocated learning through, practical experiences and critical thinking. In 1997, the Seventh-day Adventist Church recommended problem-based learning (PBL) as a preferred teaching practice for its North American K-12 schools. However, Brantley and Ruiz observed that many Seventh-day Adventist educators feel inadequate to use this method of instruction. Little information exists anywhere concerning teachers’ awareness and perceptions o f problem-based learning (PBL) or factors related to its use. This study examined the relationship between PBL, philosophy of teaching, preferences for PBL teaching components, and perceived barriers to PBL adoption and use. Method An ex post facto survey was conducted among a convenience sample of 315 K-12 teachers in 50 schools in Florida. Four instruments were used to gather data to answer four research questions. The same instruments were administered to a group of experienced PBL teachers and results were compared to the Adventist group. Results The majority of Seventh-day Adventist K-12 teachers in Florida are unaware of problem-based learning (PBL). Teachers who embrace a student-centered teaching preference are more likely to be aware of PBL. Little more than half the teachers have a student-centered teaching philosophy, and less than half appreciate the student-centered teaching components of PBL. Teaching philosophy is related to the teachers’ age and preference for PBL teaching components. More female than male teachers embrace the student-centered components ofPBL. The greatest perceived barriers to teacher implementation ofPBL included (1) assessing and reporting student learning, (2) allowing students’ needs and interests to determine pace and content of curriculum coverage, (3) a loosely structured, sometimes noisy learning environment, and (4) system unwillingness to provide PBL support sources. The majority o f the teachers did not identify factors that would deter them from implementing problem-based learning. Conclusions Although most Seventh-day Adventist teachers are unaware ofPBL and seem to embrace a teacher-centered teaching philosophy, they appear willing to learn about the method and to implement it in their classrooms. However, they do not expect support from their school systems, parents, and colleagues, as preconditions to successful adoption. It should be noted that the major barriers to PBL adoption appear to be reflective of the teaching philosophy of the school systems, parents, and teachers. Addressing these barriers is likely to increase the possibility that successful adoption will take place