Learning the selection of actions for an autonomous social robot by reinforcement learning based on motivations

Autonomy is a prime issue on robotics field and it is closely related to decision making. Last researches on decision making for social robots are focused on biologically inspired mechanisms for taking decisions. Following this approach, we propose a motivational system for decision making, using internal (drives) and external stimuli for learning to choose the right action. Actions are selected from a finite set of skills in order to keep robot's needs within an acceptable range. The robot uses reinforcement learning in order to calculate the suitability of every action in each state. The state of the robot is determined by the dominant motivation and its relation to the objects presents in its environment. The used reinforcement learning method exploits a new algorithm called Object Q-Learning. The proposed reduction of the state space and the new algorithm considering the collateral effects (relationship between different objects) results in a suitable algorithm to be applied to robots living in real environments. In this paper, a first implementation of the decision making system and the learning process is implemented on a social robot showing an improvement in robot's performance. The quality of its performance will be determined by observing the evolution of the robot's wellbeing.The funds provided by the Spanish Government through the project called “Peer to Peer Robot-Human Interaction” (R2H), of MEC (Ministry of Science and Education), the project “A new approach to social robotics” (AROS), of MICINN (Ministry of Science and Innovation), and the RoboCity2030-II-CM project (S2009/DPI-1559), funded by Programas de Actividades I+D en la Comunidad de Madrid and cofunded by Structural Funds of the EU

Avaliação participativa de tecnologia e sustentabilidade organizacional

Baseado no texto apresentado ao X Colóquio de Sociologia "Organizações e sustentabilidade", organizado pela Universidade do Minho em 2011, em BragaA Avaliação de Tecnologia (AT) toma em consideração o conhecimento acerca dos (possíveis ou prováveis) efeitos das tecnologias nos processos de tomada de decisão e a exploração de riscos tecnológicos potenciais, com efeitos secundários. Além disso, é um processo científico com o objectivo de contribuir para a formação da opinião pública e política relativa aos aspectos sociais em ciência e tecnologia. Essa formação é feita de modo interactivo e comunicativo, ultrapassando problemas de legitimidade e conflitos tecnológicos. A AT diz respeito a um processo político, seja ele relacionado com a decisão a nível parlamentar sobre a introdução ou limitação de novas tecnologias, seja a nível dos processos de participação das entidades interessadas na esfera do trabalho. Neste estudo, conclui-se que os processos de AT ao nível da organização do trabalho podem ter como objectivo atingir maiores níveis de produtividade e de desempenho do equipamento instalado, ou mesmo melhorar a qualidade do produto ou do processo produtivo. Isso não se traduz necessariamente em melhorias de rendimento ou de desempenho dos próprios empregados ou trabalhadores, pelo que a participação destes actores é também fundamental neste processo

Human Intent Prediction Using Markov Decision Processes

Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/97080/1/AIAA2012-2445.pd

Exploring Natural User Abstractions For Shared Perceptual Manipulator Task Modeling & Recovery

State-of-the-art domestic robot assistants are essentially autonomous mobile manipulators capable of exerting human-scale precision grasps. To maximize utility and economy, non-technical end-users would need to be nearly as efficient as trained roboticists in control and collaboration of manipulation task behaviors. However, it remains a significant challenge given that many WIMP-style tools require superficial proficiency in robotics, 3D graphics, and computer science for rapid task modeling and recovery. But research on robot-centric collaboration has garnered momentum in recent years; robots are now planning in partially observable environments that maintain geometries and semantic maps, presenting opportunities for non-experts to cooperatively control task behavior with autonomous-planning agents exploiting the knowledge. However, as autonomous systems are not immune to errors under perceptual difficulty, a human-in-the-loop is needed to bias autonomous-planning towards recovery conditions that resume the task and avoid similar errors. In this work, we explore interactive techniques allowing non-technical users to model task behaviors and perceive cooperatively with a service robot under robot-centric collaboration. We evaluate stylus and touch modalities that users can intuitively and effectively convey natural abstractions of high-level tasks, semantic revisions, and geometries about the world. Experiments are conducted with \u27pick-and-place\u27 tasks in an ideal \u27Blocks World\u27 environment using a Kinova JACO six degree-of-freedom manipulator. Possibilities for the architecture and interface are demonstrated with the following features; (1) Semantic \u27Object\u27 and \u27Location\u27 grounding that describe function and ambiguous geometries (2) Task specification with an unordered list of goal predicates, and (3) Guiding task recovery with implied scene geometries and trajectory via symmetry cues and configuration space abstraction. Empirical results from four user studies show our interface was much preferred than the control condition, demonstrating high learnability and ease-of-use that enable our non-technical participants to model complex tasks, provide effective recovery assistance, and teleoperative control

Human system modelling in support of manufacturing enterprise design and change

Organisations comprise human and technical systems that typically perform a variety of business, engineering and production roles. Human systems comprise individuals, people groups and teams that work systematically to conceive, implement, develop and manage the purposes of any enterprise in response to customer requirements. Recently attention has been paid to modelling aspects of people working within production systems, with a view to improving: production performance, effective resource allocation and optimum resource management. In the research reported, graphical and computer executable models of people have been conceived and used in support of human systems engineering. The approach taken has been to systematically decompose and represent processes so that elemental production and management activities can be modelled as explicit descriptions of roles that human systems can occupy as role holders. First of all, a preliminary modelling method (MM1) was proposed for modelling human systems in support of engineering enterprise; then MM1 was implemented and tested in a case study company 1. Based on findings of this exploratory research study an improved modelling method (MM2) was conceived and instrumented. Here characterising customer related product dynamic impacts extended MM1 modelling concepts and methods and related work system changes. MM2 was then tested in case study company 2 to observe dynamic behaviours of selected system models derived from actual company knowledge and data. Case study 2 findings enabled MM2 to be further improved leading to MM3. MM3 improvements stem from the incorporation of so-called DPU (Dynamic Producer Unit) concepts, related to the modelling of human and technical resource system components . Case study 4 models a human system for targeted users i.e. production managers etc to facilitate analysis of human configuration and also cost modelling. Modelling approaches MM2, MM3 and also Case Study 4 add to knowledge about ways of facilitating quantitative analysis and comparison between different human system configurations. These new modelling methods allow resource system behaviours to be matched to specific, explicitly defined, process-oriented requirements drawn from manufacturing workplaces currently operating in general engineering, commercial furniture and white goods industry sectors

Deep Learning for Decision Making and Autonomous Complex Systems

Deep learning consists of various machine learning algorithms that aim to learn multiple levels of abstraction from data in a hierarchical manner. It is a tool to construct models using the data that mimics a real world process without an exceedingly tedious modelling of the actual process. We show that deep learning is a viable solution to decision making in mechanical engineering problems and complex physical systems. In this work, we demonstrated the application of this data-driven method in the design of microfluidic devices to serve as a map between the user-defined cross-sectional shape of the flow and the corresponding arrangement of micropillars in the flow channel that contributed to the flow deformation. We also present how deep learning can be used in the early detection of combustion instability for prognostics and health monitoring of a combustion engine, such that appropriate measures can be taken to prevent detrimental effects as a result of unstable combustion. One of the applications in complex systems concerns robotic path planning via the systematic learning of policies and associated rewards. In this context, a deep architecture is implemented to infer the expected value of information gained by performing an action based on the states of the environment. We also applied deep learning-based methods to enhance natural low-light images in the context of a surveillance framework and autonomous robots. Further, we looked at how machine learning methods can be used to perform root-cause analysis in cyber-physical systems subjected to a wide variety of operation anomalies. In all studies, the proposed frameworks have been shown to demonstrate promising feasibility and provided credible results for large-scale implementation in the industry

Human Autonomy Teaming - The Teamwork of the Future

Dies ist ein Herausgeberwerk.Der Zusammenarbeit von Mensch und Technik kommt angesichts technologischer Fortschritte eine immer größere Bedeutung zu. Das Human Autonomy Teaming (HAT) birgt in diesem Zusammenhang als neue Form der Teamarbeit zwischen menschlichen Teammitgliedern und technischen Einheiten, sogenannten autonomen Agenten, ein großes Potenzial. Der Mensch kooperiert mit seinem technischen Teammitglied und wird von diesem bei gemeinsamen Aufgaben im Team unterstützt. Beide Akteure ergänzen sich mit ihren individuellen Stärken gegenseitig im Team. In diesem Buch sind aktuelle Themen im Rahmen des HAT für Forscher/innen und Praktiker/innen übersichtlich aufbereitet, um gemeinsam zur erfolgreichen Umsetzung autonomer Agenten als Teammitglied des Menschen im Sinne eines HAT beitragen zu können. In Kapitel 1 wird in das Thema eingeleitet, grundlegende Definitionen und Modelle für das gesamte Werk vorgestellt sowie die Potentiale des HAT aufgezeigt. Kapitel 2 thematisiert menschliche und technische Anforderungen für erfolgreiches HAT, bevor in Kapitel 3 näher auf die Zusammenarbeit zwischen Mensch und Technik und die damit einhergehenden Stärken und Schwächen eingegangen wird. Kapitel 4 liefert Einblicke in aktuelle Anwendungsgebiete des HAT. Abschließend werden in Kapitel 5 zukünftige Entwicklungen des HAT diskutiert. As a result of technological advances, collaboration between humans and technology is becoming increasingly important. In this context, Human Autonomy Teaming (HAT), as a new form of teamwork between humans and technology, so-called autonomous agents, has great potential and offers many possibilities in research and application. Both team members complement each other with their individual strengths striving to achieve a common goal. In this book, current topics within the framework of the HAT are clearly presented for researchers and practitioners in order to be able to jointly contribute to the successful implementation of autonomous agents as team members in the sense of HAT. Chapter 1 introduces the topic, basic definitions and models for the entire work, and shows the potential of HAT. Chapter 2 deals with human and technological requirements for successful HAT, before chapter 3 goes into more detail on the cooperation between humans and technology and the associated strengths and weaknesses. Chapter 4 provides insights into current fields of application of HAT. Finally, in Chapter 5, future developments of HAT are discussed

Safe and Efficient Robot Action Choice Using Human Intent Prediction in Physically-Shared Space Environments.

Emerging robotic systems are capable of autonomously planning and executing well-defined tasks, particularly when the environment can be accurately modeled. Robots supporting human space exploration must be able to safely interact with human astronaut companions during intravehicular and extravehicular activities. Given a shared workspace, efficiency can be gained by leveraging robotic awareness of its human companion. This dissertation presents a modular architecture that allows a human and robotic manipulator to efficiently complete independent sets of tasks in a shared physical workspace without the robot requiring oversight or situational awareness from its human companion. We propose that a robot requires four capabilities to act safely and optimally with awareness of its companion: sense the environment and the human within it; translate sensor data into a form useful for decision-making; use this data to predict the human’s future intent; and then use this information to inform its action-choice based also on the robot’s goals and safety constraints. We first present a series of human subject experiments demonstrating that human intent can help a robot predict and avoid conflict, and that sharing the workspace need not degrade human performance so long as the manipulator does not distract or introduce conflict. We describe an architecture that relies on Markov Decision Processes (MDPs) to support robot decision-making. A key contribution of our architecture is its decomposition of the decision problem into two parts: human intent prediction (HIP) and robot action choice (RAC). This decomposition is made possible by an assumption that the robot’s actions will not influence human intent. Presuming an observer that can feedback human actions in real-time, we leverage the well-known space environment and task scripts astronauts rehearse in advance to devise models for human intent prediction and robot action choice. We describe a series of case studies for HIP and RAC using a minimal set of state attributes, including an abbreviated action-history. MDP policies are evaluated in terms of model fitness and safety/efficiency performance tradeoffs. Simulation results indicate that incorporation of both observed and predicted human actions improves robot action choice. Future work could extend to more general human-robot interaction.PhDAerospace EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/107160/1/cmcghan_1.pd