Multicore and FPGA implementations of emotional-based agent architectures

The final publication is available at Springer via http://dx.doi.org/10.1007/s11227-014-1307-6.Control architectures based on Emotions are becoming promising solutions for the implementation of future robotic agents. The basic controllers of the architecture are the emotional processes that decide which behaviors of the robot must activate to fulfill the objectives. The number of emotional processes increases (hundreds of millions/s) with the complexity level of the application, reducing the processing capacity of the main processor to solve complex problems (millions of decisions in a given instant). However, the potential parallelism of the emotional processes permits their execution in parallel on FPGAs or Multicores, thus enabling slack computing in the main processor to tackle more complex dynamic problems. In this paper, an emotional architecture for mobile robotic agents is presented. The workload of the emotional processes is evaluated. Then, the main processor is extended with FPGA co-processors through Ethernet link. The FPGAs will be in charge of the execution of the emotional processes in parallel. Different Stratix FPGAs are compared to analyze their suitability to cope with the proposed mobile robotic agent applications. The applications are set up taking into account different environmental conditions, robot dynamics and emotional states. Moreover, the applications are run also on Multicore processors to compare their performance in relation to the FPGAs. Experimental results show that Stratix IV FPGA increases the performance in about one order of magnitude over the main processor and solves all the considered problems. Quad-Core increases the performance in 3.64 times, allowing to tackle about 89 % of the considered problems. Quad-Core has a lower cost than a Stratix IV, so more adequate solution but not for the most complex application. Stratix III could be applied to solve problems with around the double of the requirements that the main processor could support. Finally, a Dual-Core provides slightly better performance than stratix III and it is relatively cheaper.This work was supported in part under Spanish Grant PAID/2012/325 of "Programa de Apoyo a la Investigacion y Desarrollo. Proyectos multidisciplinares", Universitat Politecnica de Valencia, Spain.Domínguez Montagud, CP.; Hassan Mohamed, H.; Crespo, A.; Albaladejo Meroño, J. (2015). Multicore and FPGA implementations of emotional-based agent architectures. Journal of Supercomputing. 71(2):479-507. https://doi.org/10.1007/s11227-014-1307-6S479507712Malfaz M, Salichs MA (2010) Using MUDs as an experimental platform for testing a decision making system for self-motivated autonomous agents. Artif Intell Simul Behav J 2(1):21–44Damiano L, Cañamero L (2010) Constructing emotions. Epistemological groundings and applications in robotics for a synthetic approach to emotions. In: Proceedings of international symposium on aI-inspired biology, The Society for the Study of Artificial Intelligence, pp 20–28Hawes N, Wyatt J, Sloman A (2009) Exploring design space for an integrated intelligent system. Knowl Based Syst 22(7):509–515Sloman A (2009) Some requirements for human-like robots: why the recent over-emphasis on embodiment has held up progress. Creat Brain Like Intell 2009:248–277Arkin RC, Ulam P, Wagner AR (2012) Moral decision-making in autonomous systems: enforcement, moral emotions, dignity, trust and deception. In: Proceedings of the IEEE, Mar 2012, vol 100, no 3, pp 571–589iRobot industrial robots website. http://www.irobot.com/gi/ground/ . Accessed 22 Sept 2014Moravec H (2009) Rise of the robots: the future of artificial intelligence. Scientific American, March 2009. http://www.scientificamerican.com/article/rise-of-the-robots/ . Accessed 14 Oct 2014.Thu Bui L, Abbass HA, Barlow M, Bender A (2012) Robustness against the decision-maker’s attitude to risk in problems with conflicting objectives. IEEE Trans Evolut Comput 16(1):1–19Pedrycz W, Song M (2011) Analytic hierarchy process (AHP) in group decision making and its optimization with an allocation of information granularity. IEEE Trans Fuzzy Syst 19(3):527–539Lee-Johnson CP, Carnegie DA (2010) Mobile robot navigation modulated by artificial emotions. IEEE Trans Syst Man Cybern Part B 40(2):469–480Daglarli E, Temeltas H, Yesiloglu M (2009) Behavioral task processing for cognitive robots using artificial emotions. Neurocomputing 72(13):2835–2844Ventura R, Pinto-Ferreira C (2009) Responding efficiently to relevant stimuli using an emotion-based agent architecture. Neurocomputing 72(13):2923–2930Arkin RC, Ulam P, Wagner AR (2012) Moral decision-making in autonomous systems: enforcement, moral emotions, dignity, trust and deception. Proc IEEE 100(3):571–589Salichs MA, Malfaz M (2012) A new approach to modeling emotions and their use on a decision-making system for artificial agents. Affect Comput IEEE Trans 3(1):56–68Altera Corporation (2011) Stratix III device handbook, vol 1–2, version 2.2. http://www.altera.com/literature/lit-stx3.jsp . Accessed 14 Oct 2014.Altera Corporation (2014) Stratix IV device handbook, vol 1–4, version 5.9. http://www.altera.com/literature/lit-stratix-iv.jsp . Accessed 14 Oct 2014.Naouar MW, Monmasson E, Naassani AA, Slama-Belkhodja I, Patin N (2007) FPGA-based current controllers for AC machine drives: a review. IEEE Trans Ind Electr 54(4):1907–1925Intel Corporation (2014) Desktop 4th generation Intel Core Processor Family, Desktop Intel Pentium Processor Family, and Desktop Intel Celeron Processor Family, Datasheet, vol 1, 2March JL, Sahuquillo J, Hassan H, Petit S, Duato J (2011) A new energy-aware dynamic task set partitioning algorithm for soft and hard embedded real-time systems. Comput J 54(8):1282–1294Del Campo I, Basterretxea K, Echanobe J, Bosque G, Doctor F (2012) A system-on-chip development of a neuro-fuzzy embedded agent for ambient-intelligence environments. IEEE Trans Syst Man Cybern Part B 42(2):501–512Pedraza C, Castillo J, Martínez JI, Huerta P, Bosque JL, Cano J (2011) Genetic algorithm for Boolean minimization in an FPGA cluster. J Supercomput 58(2):244–252Orlowska-Kowalska T, Kaminski M (2011) FPGA implementation of the multilayer neural network for the speed estimation of the two-mass drive system. IEEE Trans Ind Inf 7(3):436–445Cassidy AS, Merolla P, Arthur JV, Esser SK, Jackson B, Alvarez-icaza R, Datta P, Sawada J, Wong TM, Feldman V, Amir A, Ben-dayan D, Mcquinn E, Risk WP, Modha DS (2013) Cognitive computing building block: a versatile and efficient digital neuron model for neurosynaptic cores. In: Proceedings of international joint conference on neural networks, IEEE (IJCNN’2013)IBM Cognitive Computing and Neurosynaptic chips website. http://www.research.ibm.com/cognitive-computing/neurosynaptic-chips.shtml . Accessed 22 Sept 2014Seo E, Jeong J, Park S, Lee J (2008) Energy efficient scheduling of real-time tasks on multicore processors. IEEE Trans Parallel Distrib Syst 19(11):1540–1552Lehoczky J, Sha L, Ding Y (1989) The rate monotonic scheduling algorithm: exact characterization and average case behavior. In: Proceedings of real time systems symposium, IEEE 1989, pp 166–171Ng-Thow-Hing V, Lim J, Wormer J, Sarvadevabhatla RK, Rocha C, Fujimura K, Sakagami Y (2008) The memory game: creating a human-robot interactive scenario for ASIMO. In: Proceedings of intelligent robots and systems, 2008, IROS 2008, IEEE/RSJ international conference, pp 779–78

A Hierarchical Emotion Regulated Sensorimotor Model: Case Studies

Inspired by the hierarchical cognitive architecture and the perception-action model (PAM), we propose that the internal status acts as a kind of common-coding representation which affects, mediates and even regulates the sensorimotor behaviours. These regulation can be depicted in the Bayesian framework, that is why cognitive agents are able to generate behaviours with subtle differences according to their emotion or recognize the emotion by perception. A novel recurrent neural network called recurrent neural network with parametric bias units (RNNPB) runs in three modes, constructing a two-level emotion regulated learning model, was further applied to testify this theory in two different cases.Comment: Accepted at The 5th International Conference on Data-Driven Control and Learning Systems. 201

A Review of Verbal and Non-Verbal Human-Robot Interactive Communication

In this paper, an overview of human-robot interactive communication is presented, covering verbal as well as non-verbal aspects of human-robot interaction. Following a historical introduction, and motivation towards fluid human-robot communication, ten desiderata are proposed, which provide an organizational axis both of recent as well as of future research on human-robot communication. Then, the ten desiderata are examined in detail, culminating to a unifying discussion, and a forward-looking conclusion

Human inspired robotic path planning and heterogeneous robotic mapping

One of the biggest challenges facing robotics is the ability for a robot to autonomously navigate real-world unknown environments and is considered by many to be a key prerequisite of truly autonomous robots. Autonomous navigation is a complex problem that requires a robot to solve the three problems of navigation: localisation, goal recognition, and path-planning. Conventional approaches to these problems rely on computational techniques that are inherently rigid and brittle. That is, the underlying models cannot adapt to novel input, nor can they account for all potential external conditions, which could result in erroneous or misleading decision making. In contrast, humans are capable of learning from their prior experiences and adapting to novel situations. Humans are also capable of sharing their experiences and knowledge with other humans to bootstrap their learning. This is widely thought to underlie the success of humanity by allowing high-fidelity transmission of information and skills between individuals, facilitating cumulative knowledge gain. Furthermore, human cognition is influenced by internal emotion states. Historically considered to be a detriment to a person's cognitive process, recent research is regarding emotions as a beneficial mechanism in the decision making process by facilitating the communication of simple, but high-impact information. Human created control approaches are inherently rigid and cannot account for the complexity of behaviours required for autonomous navigation. The proposed thesis is that cognitive inspired mechanisms can address limitations in current robotic navigation techniques by allowing robots to autonomously learn beneficial behaviours from interacting with its environment. The first objective is to enable the sharing of navigation information between heterogeneous robotic platforms. The second objective is to add flexibility to rigid path-planning approaches by utilising emotions as low-level but high-impact behavioural responses. Inspired by cognitive sciences, a novel cognitive mapping approach is presented that functions in conjunction with current localisation techniques. The cognitive mapping stage utilises an Anticipatory Classifier System (ACS) to learn the novel Cognitive Action Map (CAM) of decision points, areas in which a robot must determine its next action (direction of travel). These physical actions provide a shared means of understanding the environment to allow for communicating learned navigation information. The presented cognitive mapping approach has been trained and evaluated on real-world robotic platforms. The results show the successful sharing of navigation information between two heterogeneous robotic platforms with different sensing capabilities. The results have also demonstrated the novel contribution of autonomously sharing navigation information between a range-based (GMapping) and vision-based (RatSLAM) localisation approach for the first time. The advantage of sharing information between localisation techniques allows an individual robotic platform to utilise the best fit localisation approach for its sensors while still being able to provide useful navigation information for robots with different sensor types. Inspired by theories on natural emotions, this work presents a novel emotion model designed to improve a robot's navigation performance through learning to adapt a rigid path-planning approach. The model is based on the concept of a bow-tie structure, linking emotional reinforcers and behavioural modifiers through intermediary emotion states. An important function of the emotions in the model is to provide a compact set of high-impact behaviour adaptations, reducing an otherwise tangled web of stimulus-response patterns. Crucially, the system learns these emotional responses with no human pre-specifying the behaviour of the robot, hence avoiding human bias. The results of training the emotion model demonstrate that it is capable of learning up to three emotion states for robotic navigation without human bias: fear, apprehension, and happiness. The fear and apprehension responses slow the robot's speed and drive the robot away from obstacles when the robot experiences pain, or is uncertain of its current position. The happiness response increases the speed of the robot and reduces the safety margins around obstacles when pain is absent, allowing the robot to drive closer to obstacles. These learned emotion responses have improved the navigation performance of the robot by reducing collisions and navigation times, in both simulated and real-world experiments. The two emotion model (fear and happiness) improved performance the most, indicating that a robot may only require two emotion states (fear and happiness) for navigation in common, static domains

Emotion as an emergent phenomenon of the neurocomputational energy regulation mechanism of a cognitive agent in a decision-making task:

Biological agents need to complete perception-action cycles to perform various cognitive and biological tasks such as maximizing their wellbeing and their chances of genetic continuation. However, the processes performed in these cycles come at a cost. Such costs force the agent to evaluate a tradeoff between the optimality of the decision making and the time and computational effort required to make it. Several cognitive mechanisms that play critical roles in managing this tradeoff have been identified. These mechanisms include adaptation, learning, memory, attention, and planning. One of the often overlooked outcomes of these cognitive mechanisms, in spite of the critical effect that they may have on the perception-action cycle of organisms, is "emotion." In this study, we hold that emotion can be considered as an emergent phenomenon of a plausible neurocomputational energy regulation mechanism, which generates an internal reward signal to minimize the neural energy consumption of a sequence of actions (decisions), where each action triggers a visual memory recall process. To realize an optimal action selection over a sequence of actions in a visual recalling task, we adopted a model-free reinforcement learning framework, in which the reward signal—that is, the cost—was based on the iteration steps of the convergence state of an associative memory network. The proposed mechanism has been implemented in simulation and on a robotic platform: the iCub humanoid robot. The results show that the computational energy regulation mechanism enables the agent to modulate its behavior to minimize the required neurocomputational energy in performing the visual recalling task

Neurobiologically Inspired Mobile Robot Navigation and Planning

After a short review of biologically inspired navigation architectures, mainly relying on modeling the hippocampal anatomy, or at least some of its functions, we present a navigation and planning model for mobile robots. This architecture is based on a model of the hippocampal and prefrontal interactions. In particular, the system relies on the definition of a new cell type “transition cells” that encompasses traditional “place cells”

Emotion-based Parameter Modulation for a Mobile Robot Planning and Control System

The hypothesis that artificial emotion-like mechanisms can improve the adaptive performance of robots and intelligent systems has gained considerable support in recent years. While artificial emotions are typically employed to facilitate human-machine interaction, this thesis instead focuses on modelling emotions and affect in a non-social context. In particular, affective mechanisms are applied to the problem of mobile robot navigation. A three-layered reactive/deliberative controller is developed and implemented, resulting in several contributions to the field of mobile robot control. Rather than employing a reactive layer, a deliberative layer and an interface between them, the control problem is decomposed into three different conceptual spaces - position space, direction space and velocity space - with a distinct control layer applied to each. Existing directional and velocity space approaches such as the vector field histogram (VFH) and dynamic window methods employ different underlying mechanisms and terminology. This thesis unifies these approaches in order to compare and combine them. The weighted sum objective functions employed by some existing approaches that inspired the presented directional and velocity control layers are replaced by weighted products. This enables some hard constraints to be relaxed in favour of weighted contributions, potentially improving a system's flexibility without sacrificing safety (but coming at a cost to efficiency). An affect model is developed that conceptualises emotions and other affective interactions as modulations of cognitive processes. Unlike other models of affect-modulated cognition (e.g. Dorner and Hille, 1995), this model is designed specifically to address problems relating to mobile robot navigation. The role of affect in this model is to continuously adapt a controller's behaviour patterns in response to different environments and momentary conditions encountered by the robot. Affective constructs such as moods and emotions are represented as intensity values that arise from hard-coded interpretations of local stimuli, as well as from learned associations stored in global maps. They are expressed as modulations of control parameters and location-specific biases to path-planning. Extensive simulation experiments are conducted in procedurally-generated environments to assess the performance contributions of this model and its individual components