A socio-technical approach for assistants in human-robot collaboration in industry 4.0

The introduction of technologies disruptive of Industry 4.0 in the workplace integrated through human cyber-physical systems causes operators to face new challenges. These are reflected in the increased demands presented in the operator's capabilities physical, sensory, and cognitive demands. In this research, cognitive demands are the most interesting. In this perspective, assistants are presented as a possible solution, not as a tool but as a set of functions that amplify human capabilities, such as exoskeletons, collaborative robots for physical capabilities, virtual and augmented reality for sensory capabilities. Perhaps chatbots and softbots for cognitive capabilities, then the need arises to ask ourselves: How can operator assistance systems 4.0 be developed in the context of industrial manufacturing? In which capacities does the operator need more assistance? From the current paradigm of systematization, different approaches are used within the context of the workspace in industry 4.0. Thus, the functional resonance analysis method (FRAM) is used to model the workspace from the sociotechnical system approach, where the relationships between the components are the most important among the functions to be developed by the human-robot team. With the use of simulators for both robots and robotic systems, the behavior of the variability of the human-robot team is analyzed. Furthermore, from the perspective of cognitive systems engineering, the workspace can be studied as a joint cognitive system, where cognition is understood as distributed, in a symbiotic relationship between the human and technological agents. The implementation of a case study as a human-robot collaborative workspace allows evaluating the performance of the human-robot team, the impact on the operator's cognitive abilities, and the level of collaboration achieved in the human-robot team through a set of metrics and proven methods in other areas, such as cognitive systems engineering, human-machine interaction, and ergonomics. We conclude by discussing the findings and outlook regarding future research questions and possible developments.La introducción de tecnologías disruptivas de Industria 4.0 en el lugar de trabajo integradas a través de sistemas ciberfísicos humanos hace que los operadores enfrenten nuevos desafíos. Estos se reflejan en el aumento de las demandas presentadas en las capacidades físicas, sensoriales y cognitivas del operador. En esta investigación, las demandas cognitivas son las más interesantes. En esta perspectiva, los asistentes se presentan como una posible solución, no como una herramienta sino como un conjunto de funciones que amplifican las capacidades humanas, como exoesqueletos, robots colaborativos para capacidades físicas, realidad virtual y aumentada para capacidades sensoriales. Quizás chatbots y softbots para capacidades cognitivas, entonces surge la necesidad de preguntarnos: ¿Cómo se pueden desarrollar los sistemas de asistencia al operador 4.0 en el contexto de la fabricación industrial? ¿En qué capacidades el operador necesita más asistencia? A partir del paradigma actual de sistematización, se utilizan diferentes enfoques dentro del contexto del espacio de trabajo en la industria 4.0. Así, se utiliza el método de análisis de resonancia funcional (FRAM) para modelar el espacio de trabajo desde el enfoque del sistema sociotécnico, donde las relaciones entre los componentes son las más importantes entre las funciones a desarrollar por el equipo humano-robot. Con el uso de simuladores tanto para robots como para sistemas robóticos se analiza el comportamiento de la variabilidad del equipo humano-robot. Además, desde la perspectiva de la ingeniería de sistemas cognitivos, el espacio de trabajo puede ser estudiado como un sistema cognitivo conjunto, donde la cognición se entiende distribuida, en una relación simbiótica entre los agentes humanos y tecnológicos. La implementación de un caso de estudio como un espacio de trabajo colaborativo humano-robot permite evaluar el desempeño del equipo humano-robot, el impacto en las habilidades cognitivas del operador y el nivel de colaboración alcanzado en el equipo humano-robot a través de un conjunto de métricas y métodos probados en otras áreas, como la ingeniería de sistemas cognitivos, la interacción hombre-máquina y la ergonomía. Concluimos discutiendo los hallazgos y las perspectivas con respecto a futuras preguntas de investigación y posibles desarrollos.Postprint (published version

Combining task and motion planning for mobile manipulators

Aplicat embargament des de la data de defensa fins el dia 31/12/2019Premi Extraordinari de Doctorat, promoció 2018-2019. Àmbit d’Enginyeria IndustrialThis thesis addresses the combination of task and motion planning which deals with different types of robotic manipulation problems. Manipulation problems are referred to as mobile manipulation, collaborative multiple mobile robots tasks, and even higher dimensional tasks (like bi-manual robots or mobile manipulators). Task and motion planning problems needs to obtain a geometrically feasible manipulation plan through symbolic and geometric search space. The combination of task and motion planning levels has emerged as a challenging issue as the failure leads robots to dead-end tasks due to geometric constraints. In addition, task planning is combined with physics-based motion planning and information to cope with manipulation tasks in which interactions between robots and objects are required, or also a low-cost feasible plan in terms of power is looked for. Moreover, combining task and motion planning frameworks is enriched by introducing manipulation knowledge. It facilitates the planning process and aids to provide the way of executing symbolic actions. Combining task and motion planning can be considered under uncertain information and with human-interaction. Uncertainty can be viewed in the initial state of the robot world or the result of symbolic actions. To deal with such issues, contingent-based task and motion planning is proposed using a perception system and human knowledge. Also, robots can ask human for those tasks which are difficult or infeasible for the purpose of collaboration. An implementation framework to combine different types of task and motion planning is presented. All the required modules and tools are also illustrated. As some task planning algorithms are implemented in Prolog or C++ languages and our geometric reasoner is developed in C++, the flow of information between different languages is explained.Aquesta tesis es centra en les eines de planificació combinada a nivell de tasca i a nivell de moviments per abordar diferents problemes de manipulació robòtica. Els problemes considerats són de navegació de robots mòbil enmig de obstacles no fixes, tasques de manipulació cooperativa entre varis robots mòbils, i tasques de manipulació de dimensió més elevada com les portades a terme amb robots bi-braç o manipuladors mòbils. La planificació combinada de tasques i de moviments ha de cercar un pla de manipulació que sigui geomètricament realitzable, a través de d'un espai de cerca simbòlic i geomètric. La combinació dels nivells de planificació de tasca i de moviments ha sorgit com un repte ja que les fallades degudes a les restriccions geomètriques poden portar a tasques sense solució. Addicionalment, la planificació a nivell de tasca es combina amb informació de la física de l'entorn i amb mètodes de planificació basats en la física, per abordar tasques de manipulació en les que la interacció entre el robot i els objectes és necessària, o també si es busca un pla realitzable i amb un baix cost en termes de potència. A més, el marc proposat per al combinació de la planificació a nivell de tasca i a nivell de moviments es millora mitjançant l'ús de coneixement, que facilita el procés de planificació i ajuda a trobar la forma d'executar accions simbòliques. La combinació de nivells de planificació també es pot considerar en casos d'informació incompleta i en la interacció humà-robot. La incertesa es considera en l'estat inicial i en el resultat de les accions simbòliques. Per abordar aquest problema, es proposa la planificació basada en contingències usant un sistema de percepció i el coneixement de l'operari humà. Igualment, els robots poden demanar col·laboració a l'operari humà per a que realitzi aquelles accions que són difícils o no realitzables pel robot. Es presenta també un marc d'implementació per a la combinació de nivells de planificació usant diferents mètodes, incloent tots els mòduls i eines necessàries. Com que alguns algorismes estan implementats en Prolog i d'altres en C++, i el mòdul de raonament geomètric proposat està desenvolupat en C++, es detalla el flux d'informació entre diferents llenguatges.Award-winningPostprint (published version

Cross Case Study of an Elementary Engineering Task

Designerly play has been identified as a fundamental component of childhood learning (Baynes, 1994; Petroski, 2003). However, as students enter grade one and beyond, the increasing academic focus has resulted in the loss of opportunities for designerly play (Zhao, 2012). At the same time, there are increasing calls to increase the number, skill, and diversity of STEM workers (Brophy, Portsmore, Klein, & Rogers, 2008). The robotics based Elementary Engineering Curriculum (Heffernan, 2013) - used by students in this study - and other similar projects have the potential to increase the STEM pipeline but elementary engineering is not well-understood. Research is needed to understand how to teach engineering to students as their cognitive, motor, and social skills rapidly develop in elementary school (Alimisis, 2012; Crismond & Adams, 2012; Mead, Thomas, & Weinberg, 2012; Penner, Giles, Lehrer, & Schauble, 1997; Roth, 1996; Schunn, 2009; Wagner, 1999). The literature review and theoretical frameworks chapters of this study determined the most relevant theoretical frameworks, engineering design process models, and existing research that is relevant to a cross-sectional case study of six grade 2 and six grade 6 elementary robotics students in the context of established K-6 elementary robotics curriculum (Heffernan, 2013). Students were videotaped doing an open-ended engineering task based on LEGO robotics using talk-aloud (Ericsson & Simon, 1993) and clinical interview (Ginsburg, 1997) techniques. The engineering design processes were analyzed and compared by age and gender. Significant differences were found in final projects and engineering design process. However, the differences were not, for the most part, related to development or gender, but were related to the complexity of the ride they tried to build and the skills and structural knowledge they brought to the task. The key factors identified consisted of three executive function process skills of cognitive flexibility, causal reasoning, and planning ability, three domain specific process skills of application of mathematics and science, engineering design process skills, and design principles of stability, scale, and the structural knowledge they had of LEGO robotics, most pointedly, LEGO connection knowledge. Implications of these findings for teachers are given

The development of assistive technology to reveal knowledge of physical world concepts in young people who have profound motor impairments.

Cognitively able children and young people who have profound motor impairments and complex communication needs (the target group or TG) face many barriers to learning, communication, personal development, physical interaction and play experiences, compared to their typically developing peers. Physical interaction (and play) are known to be important components of child development, but this group currently has few suitable ways in which to participate in these activities. Furthermore, the TG may have knowledge about real world physical concepts despite having limited physical interaction experiences but it can be difficult to reveal this knowledge and conventional assessment techniques are not suitable for this group, largely due to accessibility issues. This work presents a pilot study involving a robotics-based system intervention which enabled members of the TG to experience simulated physical interaction and was designed to identify and develop the knowledge and abilities of the TG relating to physical concepts involving temporal, spatial or movement elements. The intervention involved the participants using an eye gaze controlled robotic arm with a custom made haptic feedback device to complete a set of tasks. To address issues with assessing the TG, two new digital Assistive Technology (AT) accessible assessments were created for this research, one using static images, the other video clips. Two participants belonging to the TG took part in the study. The outcomes indicated a high level of capability in performing the tasks, with the participants exhibiting a level of knowledge and ability which was much higher than anticipated. One explanation for this finding could be that they have acquired this knowledge through past experiences and ‘observational learning’. The custom haptic device was found to be useful for assessing the participants’ sense of ‘touch’ in a way which is less invasive than conventional ‘pin-prick’ techniques. The new digital AT accessible assessments seemed especially suitable for one participant, while results were mixed for the other. This suggests that a combination of ‘traditional’ assessment and a ‘practical’ intervention assessment approach may help to provide a clearer, more rounded understanding of individuals within the TG. The work makes contributions to knowledge in the field of disability and Assistive Technology, specifically regarding: AT accessible assessments; haptic device design for the TG; the combination of robotics, haptics and eye gaze for use by the TG to interact with the physical world; a deeper understanding of the TG in general; insights into designing for and working with the TG. The work and information gathered can help therapists and education staff to identify strengths and gaps in knowledge and skills, to focus learning and therapy activities appropriately, and to change the perceptions of those who work with this group, encouraging them to broaden their expectations of the TG

Interactive technologies for preschool game-based instruction: Experiences and future challenges

This is the author’s version of a work that was accepted for publication in Entertainment Computing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Entertainment Computing, vol. 17 (2016). DOI 10.1016/j.entcom.2016.07.001.[EN] According to current kindergarten curricula, game play is an important basis for children development and it is the main driving force when designing educational activities during early childhood. This paper presents a review of the current state of the art of game technologies that support pre-kindergarten and kindergarten children development. Moreover, the most emergent technologies for developing educational games for preschool children are identified and a set of future challenges are discussed. The main goal of this work is to review the state of the art in interactive technologies which will help educators, game designers and Human-Computer Interaction (HCI) experts in the area of game-based kindergarten instruction. 2016 Elsevier B.V. All rights reserved.This work received financial support from Spanish Ministry of Economy and Competitiveness and funded by the European Development Regional Fund (EDRF-FEDER) with the project TIN2014-60077-R (SUPEREMOS). This work is also supported by a predoctoral fellowship within the FPU program from the Spanish Ministry of Education, Culture and Sports to V. Nacher (FPU14/00136) and from GVA (ACIF/2014/214) to F. Garcia-Sanjuan.Nácher-Soler, VE.; García Sanjuan, F.; Jaén Martínez, FJ. (2016). Interactive technologies for preschool game-based instruction: Experiences and future challenges. Entertainment Computing. 17:19-29. https://doi.org/10.1016/j.entcom.2016.07.001S19291

AIDA: An Assistant for Workers with Intellectual and Developmental Disabilities

Roughly 1 in 5 people in the United States have an intellectual or developmental disability (IDD), which is a substantial amount of the population. In the realm of human-robot interaction, there have been many attempts to help these individuals lead more productive and independent lives. However, many of these solutions focus on helping individuals with IDD develop social skills. For the solutions that do focus on helping people with IDD increase their work productivity, many of these involve giving the user control over a robot that augments the worker’s capabilities. In this thesis, it is posited that an autonomous agent could effectively assist workers with IDD, thereby increasing their productivity. The artificially intelligent disability assistant (AIDA) is an autonomous agent that uses social scaffolding techniques to assist workers with IDD. Before designing the system, data was gathered by observing workers with IDD perform tasks in a light manufacturing facility. To test the hypothesis, an initial Wizard-of-Oz (WoZ) experiment was conducted where subjects had to assemble a box using only either their dominant or non-dominant hand. During the experiment, subjects could ask the robot for assistance, but a human operator controlled whether the robot provided a response. After the experiment, subjects were required to complete a feedback survey. Additionally, this feedback was used to refine and build the autonomous system for AIDA. The autonomous system is composed of data collection and processing modules, a scaffolding algorithm module, and robot action output modules. This system was tested in a simulated experiment using video recordings from the initial experiment. The results of the simulated experiment provide support for the hypothesis that an autonomous agent using social scaffolding techniques can increase the productivity of workers with IDD. In the future, it is desired to test the current system in a real-time human-subjects experiment before using it to assist workers with IDD

Artificial general intelligence: Proceedings of the Second Conference on Artificial General Intelligence, AGI 2009, Arlington, Virginia, USA, March 6-9, 2009

Artificial General Intelligence (AGI) research focuses on the original and ultimate goal of AI – to create broad human-like and transhuman intelligence, by exploring all available paths, including theoretical and experimental computer science, cognitive science, neuroscience, and innovative interdisciplinary methodologies. Due to the difficulty of this task, for the last few decades the majority of AI researchers have focused on what has been called narrow AI – the production of AI systems displaying intelligence regarding specific, highly constrained tasks. In recent years, however, more and more researchers have recognized the necessity – and feasibility – of returning to the original goals of the field. Increasingly, there is a call for a transition back to confronting the more difficult issues of human level intelligence and more broadly artificial general intelligence

Proceedings of the 3rd international conference on disability, virtual reality and associated technologies (ICDVRAT 2000)

The proceedings of the conferenc

A Review of Personality in Human Robot Interactions

Personality has been identified as a vital factor in understanding the quality of human robot interactions. Despite this the research in this area remains fragmented and lacks a coherent framework. This makes it difficult to understand what we know and identify what we do not. As a result our knowledge of personality in human robot interactions has not kept pace with the deployment of robots in organizations or in our broader society. To address this shortcoming, this paper reviews 83 articles and 84 separate studies to assess the current state of human robot personality research. This review: (1) highlights major thematic research areas, (2) identifies gaps in the literature, (3) derives and presents major conclusions from the literature and (4) offers guidance for future research.Comment: 70 pages, 2 figure