Cruz, Francisco

Navarro-Guerrero, Nicolás

Solis, Miguel A.

English

Institutionelles Repositorium der Leibniz Universität Hannover

TYPE EditorialPUBLISHED 24 August 2023DOI 10.3389/fnbot.2023.1256788OPEN ACCESSEDITED AND REVIEWED BYAlois C. Knoll,Technical University of Munich, Germany*CORRESPONDENCEFrancisco Cruzf.cruz@unsw.edu.auRECEIVED 11 July 2023ACCEPTED 09 August 2023PUBLISHED 24 August 2023CITATIONCruz F, Solis MA and Navarro-Guerrero N(2023) Editorial: Cognitive inspired aspects ofrobot learning. Front. Neurorobot. 17:1256788.doi: 10.3389/fnbot.2023.1256788COPYRIGHT© 2023 Cruz, Solis and Navarro-Guerrero. Thisis an open-access article distributed under theterms of the Creative Commons AttributionLicense (CC BY). The use, distribution orreproduction in other forums is permitted,provided the original author(s) and thecopyright owner(s) are credited and that theoriginal publication in this journal is cited, inaccordance with accepted academic practice.No use, distribution or reproduction ispermitted which does not comply with theseterms.Editorial: Cognitive inspiredaspects of robot learningFrancisco Cruz1,2*, Miguel A. Solis3 andNicolás Navarro-Guerrero41School of Computer Science and Engineering, University of New South Wales, Sydney, NSW, Australia,2Escuela de Ingeniería, Universidad Central de Chile, Santiago, Chile, 3Facultad de Ingenieria, UniversidadAndres Bello, Santiago, Chile, 4L3S Research Center, Leibniz Universität Hannover, Hannover, GermanyKEYWORDSrobotics, cognitive robotics, robot learning and behavior adaptation, bio-inspiredrobotics, social robotics/HRI engineersEditorial on the Research TopicCognitive inspired aspects of robot learningRobot learning enables robots to acquire new knowledge and skills through experienceand interaction with their environment. Robot learning involves developing algorithms thatallow robots to learn autonomously, adapt to new situations, and improve their performanceover time. Using machine learning, robots can analyze large amounts of data and extractpatterns to make decisions. This approach allows robots to learn from past experiences andapply that knowledge to future tasks, ultimately enhancing their capabilities and versatility.However, although machine learning has shown great potential in robot learning, it alsofaces several challenges and limitations. One significant problem, for instance, is the issueof data scarcity. Collecting sufficient and diverse data for training robots can be complexand time-consuming (Navarro-Guerrero et al., 2023). Unlike traditional machine learningapplications where large datasets might be available, gathering data for robot learning oftenrequires physical interactions and real-world environments, which can be expensive andchallenging (Navarro-Guerrero et al., 2023).Cognitive robotics combines principles from cognitive science and robotics and offerssome advantages for robot learning and intelligence. For instance, cognitive robotics offersthe potential to create intelligent and interactive robots (Bignold et al., 2021) that canunderstand, reason, learn, and adapt like living organisms (Navarro-Guerrero et al., 2017).Combining cognitive science and robotics insights allows robots to overcome limitationsassociated with traditional robotic systems and machine learning methods (Millan-Ariaset al., 2021). Therefore, Cognitive Robotics is a paradigm for developing more sophisticatedand versatile machines capable of operating in complex real-world environments.According to Krichmar (2018), there is a close relationship between neuroroboticsand cognitive robotics. Krichmar suggests that the community should focus more ongeneral cognition rather than just specific brain areas or behaviors. He also emphasizes theimportance of coupling the brain, body, and environment and studying the brain in thecontext of naturalistic behaviors.In this Research Topic, we aimed at novel contributions to new theoretical methods andapplications in cognitively inspired aspects of robot learning, such as:Frontiers inNeurorobotics 01 frontiersin.orgCruz et al. 10.3389/fnbot.2023.1256788• Development of skills in biological systems and robots.• Principles and theories of development and learning.• Self-organizing behavior.• Models of human-human and human-robot interaction.• Human-in-the-loop reinforcement learning and applications.• Multi-robot systems with human collaboration.• Non-verbal and multimodal interaction.• Models on active learning.• Architectures for lifelong learning.• The emergence of body and affordance perception.• Bio-inspired perceptual systems.• Continual sensorimotor learning.• Models for prediction, planning, and problem-solving.• Ethics and trust in computational intelligence and robotics.• Ethical reasoning and moral uncertainty.• Social learning in humans, animals, and robots.• Explainable robotic systems.• Robot prototyping of human and animal skills.• Models of perception in biological and artificial agents.• Cognitive computing.In separate paragraphs, we describe the six acceptedcontributions, highlighting their novelty and significance inaddressing some cognitive-inspired robot learning challenges.The first paper by Li et al. introduced a novel approach torobotic grasping using a neuromorphic vision sensor. The authorsfocused on the challenge of detecting robotic grasps in a dynamiccamera view of a scene with various objects. To tackle thistask, they built a large-scale and well-annotated robotic graspingdataset called Event-Grasping, which consisted of 91 objects. Inaddition, this work proposed a spatial-temporal mixed particlefilter to track the LED-based grasp rectangles and a deep neuralnetwork architecture for grasp detection. Notably, they approachedthe angle learning problem as a classification task rather than aregression. The results of their method demonstrated a significantlyhigh accuracy when evaluated on the proposed Event-Graspingdataset.Cong et al. presented a vision-proprioception model explicitlydesigned for planar object-pushing tasks. Their model efficientlyintegrated all the necessary environmental information to enablesuccessful pushing actions. To extract task-relevant informationfrom the visual input, they employed a variational autoencoderto generate compact image representations. These representationswere combined with real-time robot state data from the hardwaresystem, including the robot’s end-effector position to form thestate representation for a Markov Decision Process. They used theSoft Actor-Critic algorithm to train the robot to push objects totarget positions within a simulation environment. This algorithmenabled the robot to learn optimal pushing policies from variousrandom initial object poses. Through their experiments, the authorsdemonstrated the effectiveness of their proposed algorithm. Itoutperformed a state-based baseline model and surpassed existingpolicies that worked directly on raw image observations. The resultsshowcased superior pushing performance achieved by their vision-proprioception model.Abel et al. conducted a study focusing on how individualsperceive robots and their actions. The researchers utilizedfunctional magnetic resonance imaging (fMRI) to analyze brainactivations of both female and male participants while observingmovements performed by either a human or a robot model,which were rated for their anthropomorphic and robotic likeness.The findings revealed that observing the human model andanthropomorphic movements led to similar activations in theposterior temporal and parietal areas associated with biologicalmotion coding. On the other hand, observing the robot modelpredominantly activated areas within the ventral stream, whilerobotic movements primarily engaged the primary and higher-order motor areas. Notably, this latter activation was primarilyobserved in female participants, while male participants displayedactivations in the posterior parietal cortex for both the robot modeland robotic movements. The overall analysis indicated that menrely on the ventro-dorsal stream, utilizing previous knowledge toanalyze movements. In contrast, female participants employed thedorso-dorsal and ventral streams to analyze movement differencesand distinguish between different models.Han et al. proposed a novel approach for addressingmultiple semantic manipulation instructions using semantic-goal-conditioned reinforcement learning. The authors introduced aconservative curiosity-driven method called mutual informationmotivation with a hybrid policy mechanism (MIHM) to addresschallenges related to uncontrollability and distraction. MIHMencompassed two main contributions. Firstly, it incorporateddecoupled mutual information-based intrinsic motivation, whichprevented the agent from being driven to explore potentiallydangerous states due to uncontrollable curiosity. Secondly,it featured a precisely trained and automatically switchedhybrid policy mechanism, effectively eliminating distractionsfrom the curiosity-driven policy and enabling optimal utilizationof exploration and exploitation. In a sparse-reward roboticmanipulation task, MIHM demonstrated the fastest learningspeed compared to four state-of-the-art curiosity-driven methods.Importantly, MIHM was the only approach among the baselines tostack three objects successfully.Abpeikar et al. conducted a study on achieving swarmingcollective motion in groups of mobile robots through an iterativetransfer learning approach. The authors employed transfer learningto enable a deep learning model to recognize and adjust stablecollective motion behaviors across multiple robot platforms. Thetransfer learner approach required only a small set of initialtraining data collected from the random movements of each robot.The proposed approach iteratively updated its knowledge base,eliminating the need for extensive data collection and avoidingthe risks associated with trial-and-error learning on physical robothardware. The method’s effectiveness was tested on two robotplatforms: simulated Pioneer 3DX robots and real Sphero BOLTrobots. The transfer learning approach enabled both platformsto adjust automatically and exhibit stable collective behaviors.Furthermore, by utilizing the knowledge-base library, the tuningprocess was both fast and accurate, enhancing the overall efficiencyof the approach.Hafez et al. focused on addressing the issue of catastrophicforgetting in deep reinforcement learning agents. They proposeda novel cognitive-inspired replay memory approach based on theGrow-When-Required self-organizing network, which resembles aFrontiers inNeurorobotics 02 frontiersin.orgCruz et al. 10.3389/fnbot.2023.1256788map-based mental model of the world. The approach organizedstored transitions into a compact network. Similar samples weremerged to reduce memory size while increasing the pair-wisedistance among samples. This consolidation process enhancedthe relevancy of each sample within the memory. The studydemonstrated that adopting a map-based experience replaytechnique significantly reduced memory usage while only resultingin minor performance decreases. This approach provided apromising solution to mitigate catastrophic forgetting in deepreinforcement learning agents.The guest editors greatly appreciate the patience, diligenceand dedicated efforts of all the reviewers, many of whom madeinsightful recommendations that helped push the quality of someof the submissions to well beyond the recommended thresholdfor acceptance into the Research Topic. We would also like tothank the Specialty Chief Editors of the Frontiers in Neuroroboticsjournal, Alois C. Knoll and Florian Röhrbein, for the opportunityto run this Research Topic, as well as the editorial staff, FionaRobertson, Valeria Corlianò, Marta Benedekova, Hannah Mellor,Naomi Wawa, Marina Solares, and Oliver Boylan for theirassistance during the collation of articles and processing of thisResearch Topic. Finally, The guest editors also thank all the authors,especially for their valuable contribution to the journal and thisResearch Topic.Author contributionsFC: Conceptualization, Writing—original draft, Writing—review and editing. NN-G: Conceptualization, Writing—originaldraft, Writing—review and editing. MS: Conceptualization,Writing—original draft, Writing—review and editing.Conflict of interestThe authors declare that the research was conducted in theabsence of any commercial or financial relationships that could beconstrued as a potential conflict of interest.Publisher’s noteAll claims expressed in this article are solely thoseof the authors and do not necessarily represent those oftheir affiliated organizations, or those of the publisher,the editors and the reviewers. Any product that may beevaluated in this article, or claim that may be made byits manufacturer, is not guaranteed or endorsed by thepublisher.ReferencesBignold, A., Cruz, F., Dazeley, R., Vamplew, P., and Foale, C. (2021).Persistent rule-based interactive reinforcement learning. Neural Comput. Appl. 1–18.doi: 10.1007/s00521-021-06466-wKrichmar, J. L. (2018). Neurorobotics-A thriving community and a promisingpathway toward intelligent cognitive robots. Front. Neurorobot. 12, 42.doi: 10.3389/fnbot.2018.00042Millan-Arias, C. C., Fernandes, B. J., Cruz, F., Dazeley, R., and Fernandes,S. (2021). A robust approach for continuous interactive actor- criticalgorithms. IEEE Access 9, 104242–104260. doi: 10.1109/ACCESS.2021.3099071Navarro-Guerrero, N., Lowe, R., and Wermter, S. (2017). Improving robot motorlearning with negatively valenced reinforcement signals. Front. Neurorobot. 11, 10.doi: 10.3389/fnbot.2017.00010Navarro-Guerrero, N., Toprak, S., Josifovski, J., and Jamone, L. (2023). Visuo-haptic object perception for robots: an overview. Auton. Robots 47, 377–403.doi: 10.1007/s10514-023-10091-yFrontiers inNeurorobotics 03 frontiersin.org

Editorial: Cognitive inspired aspects of robot learning

https://www.repo.uni-hannover.de/bitstream/123456789/16381/1/fnbot-17-1256788.pdf

Editorial: Cognitive inspired aspects of robot learning

Abstract

Similar works

Full text

Available Versions

Institutionelles Repositorium der Leibniz Universität Hannover