Efficient Parallel Implementation of Active Appearance Model Fitting Algorithm on GPU

The active appearance model (AAM) is one of the most powerful model-based object detecting and tracking methods which has been widely used in various situations. However, the high-dimensional texture representation causes very time-consuming computations, which makes the AAM difficult to apply to real-time systems. The emergence of modern graphics processing units (GPUs) that feature a many-core, fine-grained parallel architecture provides new and promising solutions to overcome the computational challenge. In this paper, we propose an efficient parallel implementation of the AAM fitting algorithm on GPUs. Our design idea is fine grain parallelism in which we distribute the texture data of the AAM, in pixels, to thousands of parallel GPU threads for processing, which makes the algorithm fit better into the GPU architecture. We implement our algorithm using the compute unified device architecture (CUDA) on the Nvidia’s GTX 650 GPU, which has the latest Kepler architecture. To compare the performance of our algorithm with different data sizes, we built sixteen face AAM models of different dimensional textures. The experiment results show that our parallel AAM fitting algorithm can achieve real-time performance for videos even on very high-dimensional textures

Developing an emotional-based application for human-agent societies

The final publication is available at Springer via http://dx.doi.org/10.1007/s00500-016-2289-5The purpose of this paper is to present an emotional-based application for human-agent societies. This kind of applications are those where virtual agents and humans coexist and interact transparently into a fully integrated environment. Specifically, the paper presents an application where humans are immersed into a system that extracts and analyzes the emotional states of a human group trying to maximize the welfare of those humans by playing the most appropriate music in every moment. This system can be used not only online, calculating the emotional reaction of people in a bar to a new song, but also in simulation, to predict the people s reaction to changes in music or in the bar layout.This work is partially supported by the MINECO/FEDER TIN2015-65515-C4-1-R and the FPI Grant AP2013-01276 awarded to Jaime-Andres Rincon.Rincón Arango, JA.; Julian Inglada, VJ.; Carrascosa Casamayor, C. (2016). Developing an emotional-based application for human-agent societies. Soft Computing. 20(11):4217-4228. https://doi.org/10.1007/s00500-016-2289-5S421742282011Ali F, Amin M (2013) The influence of physical environment on emotions, customer satisfaction and behavioural intentions in chinese resort hotel industry. In: KMITL-AGBA conference Bangkok, pp 15–17Barella A, Ricci A, Boissier O, Carrascosa C (2012) MAM5: Multi-agent model for intelligent virtual environments. In: 10th European workshop on multi-agent systems (EUMAS 2012), pp 16–30Becker-Asano C, Wachsmuth I (2010) Affective computing with primary and secondary emotions in a virtual human. Auton Agents Multi-Agent Syst 20(1):32–49. doi: 10.1007/s10458-009-9094-9Billhardt H, Julián V, Corchado J, Fernández A (2014) An architecture proposal for human-agent societies. In: Highlights of practical applications of heterogeneous multi-agent systems. The PAAMS collection, Communications in Computer and Information Science, vol 430, Springer, pp 344–357, doi: 10.1007/978-3-319-07767-3_31Billhardt H, Julián V, Corchado JM, Fernández A (2015) Human-agent societies: challenges and issues. Int J Artif Intell 13(1):28–44Broekens J (2007) Emotion and reinforcement: Affective facial expressions facilitate robot learning. Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) 4451 LNAI:113–132. doi: 10.1007/978-3-540-72348-6_6Canento F, Fred A, Silva H, Gamboa H, Lourenço A (2011) Multimodal biosignal sensor data handling for emotion recognition. In: Sensors, 2011 IEEE, pp 647–650Delac K, Grgic M, Grgic S (2005) Statistics in face recognition: analyzing probability distributions of PCA, ICA and LDA performance results. In: ISPA 2005 proceedings of the 4th international symposium on image and signal processing and analysis, 2005, pp 289–294. doi: 10.1109/ISPA.2005.195425Esparcia S, Sánchez-Anguix V, Aydogan R (2013) A negotiation approach for energy-aware room allocation systems. In: 1st Workshop on conflict resolution in decision making (COREDEMA 2013), Springer, vol 365, pp 280–291GOELEVEN E, De Raedt R, LEYMAN L, Verschuere B (2008) The karolinska directed emotional faces: a validation study. Cognit Emot 22(6):1094–1118Gouaïch A, Michel F, Guiraud Y (2005) MIC*: a deployment environment for autonomous agents. Springer, BerlinHale K, Stanney K (2002) Handbook of virtual environments: design, implementation, and applications. Human Factors and Ergonomics, Taylor and Francis, OxfordshireHan DM, Lim JH (2010) Smart home energy management system using IEEE 802.15. 4 and zigbee. Consumer Electronics, IEEE Transactions on 56(3):1403–1410. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5606276Holzapfel A, Stylianou Y (2007) A statistical approach to musical genre classification using non-negative matrix factorization. In: ICASSP 2007, IEEE international conference on, acoustics, speech and signal processing, 2007. IEEE, vol 2, pp 2–693Intille SS (2002) Designing a home of the future. IEEE Pervasive Comput 1(2):76–82Ioannou SV, Raouzaiou AT, Tzouvaras VA, Mailis TP, Karpouzis KC, Kollias SD (2005) Emotion recognition through facial expression analysis based on a neurofuzzy network. Neural Netw 18(4):423–435. doi: 10.1016/j.neunet.2005.03.004Jain D, Kobti Z (2011) Simulating the effect of emotional stress on task performance using OCC. Adv Artif Intell, Springer, pp 204–209. http://link.springer.com/chapter/10.1007/978-3-642-21043-3_24Journal I, Technological F, Singla K (2014) Audio noise reduction using different filters 1,2 1(11):1373–1375Kim MH, Joo YH, Park JB (2005) Emotion detection algorithm using frontal face image. In: International conference on computer application in shipbuildingLeon E, Clarke G, Callaghan V, Doctor F (2010) Affect-aware behaviour modelling and control inside an intelligent environment. Pervasive Mob Comput 6(5):559–574. doi: 10.1016/j.pmcj.2009.12.002Mangina E, Carbo J, Molina JM (2009) Agent-based ubiquitous computing. Atlantis Press : World Scientific, Amsterdam; Paris. doi: 10.2991/978-94-91216-31-2Masthoff J (2011) Group recommender systems: Combining individual models. Recommender systems handbook. Springer, Berlin, pp 677–702McCarthy JF, Anagnost TD (1998) Musicfx: An arbiter of group preferences for computer supported collaborative workouts. In: Proceedings of the 1998 ACM conference on computer supported cooperative work, ACM, New York, NY, USA, CSCW ’98, pp 363–372Mehrabian A (1997) Analysis of affiliation-related traits in terms of the PAD temperament model. J Psychol 131(1):101–117. doi: 10.1080/00223989709603508Ortony A (1990) The cognitive structure of emotions. Cambridge University Press, CambridgePiana S, Odone F, Verri A, Camurri A (2014) Real-time Automatic Emotion Recognition from Body Gestures. arXiv preprint pp 1–7. arXiv:1402.5047Richert W, Coelho LP (2013) Building machine learning systems with python. Packt Publishing, BirminghamRincon J, Carrascosa C, Garcia E (2014a) Developing Intelligent Virtual Environments using MAM5 Meta-Model. In: International conference on practical applications of agents and multi-agent systems, Springer, pp 379–382Rincon J, Julian V, Carrascosa C (2015) Social emotional model. In: 13th International conference on practical applications of agents and multi-agent systems, LNAI, vol 9086, pp 199–210Rincon JA, Garcia E, Julian V, Carrascosa C (2014b) Developing adaptive agents situated in intelligent virtual environments. In: International conference on hybrid artificial intelligence systems, 8480 in LNCS, Springer, pp 98–109Sánchez-Anguix V, Julian V, Botti V, García-Fornes A (2013) Studying the impact of negotiation environments on negotiation teams’ performance. Inf Sci 219:17–40Sánchez-Anguix V, Aydogan R, Julian V, Jonker C (2014) Unanimously acceptable agreements for negotiation teams in unpredictable domains. Electron Commer Res Appl 13(4):243–265Satyanarayanan M (2002) A catalyst for mobile and ubiquitous computing. Pervasive Computing, IEEE 1(1):2–5. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=993138Saunier J, Carrascosa C, Galland S, Kanmeugne PS (2015) Agent bodies: An interface between agent and environment. In: Agent environments for multi-agent systems IV, Springer, pp 25–40Scott T, Green WB, Stuart A (2005) Interactive effects of low-pass filtering and masking noise on word recognition. J Am Acad Audiol 114(11):867–878Senechal T, Rapp V, Prevost L (2011) Facial feature tracking for emotional dynamic analysis. In: Blanc-Talon, J, Kleihorst, R, Philips, W, Popescu, D, Scheunders, P (eds.) Advanced Concepts for Intelligent Vision Systems. Lecture Notes in Computer Science, Vol 6915, pp 495–506Thalmann D, Musse SR, Braun A (2007) Crowd simulation, vol 1. Springer, BerlinTsonos D, Stavropoulou P, Kouroupetroglou G, Deligiorgi D, Papatheodorou N (2014) Emotional prosodic model evaluation for greek expressive text-to-speech synthesis. Lecture Notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) 8514 LNCS(PART 2):166–174. doi: 10.1007/978-3-319-07440-5_16Tzanetakis G, Cook P (2002) Musical genre classification of audio signals. IEEE Trans Speech Audio Process 10(5):293–302. doi: 10.1109/TSA.2002.800560Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154. doi: 10.1023/B:VISI.0000013087.49260.fbVisutsak P (2012) Emotion classification using adaptive SVMs. Int J Comput Commun Eng 1(3):279–282Vukadinovic D, Pantic M (2005) Fully automatic facial feature point detection using Gabor feature based boosted classifiers. In: IEEE International conference on, systems, man and cybernetics, 2005 IEEE, vol 2, pp 1692–169

Facial feature tracking for Emotional Dynamic Analysis

Abstract. This article presents a feature-based framework to automatically track 18 facial landmarks for emotion recognition and emotional dynamic analysis. With a new way of using multi-kernel learning, we combine two methods: the first matches facial feature points between consecutive images and the second uses an offline learning of the facial landmark appearance. Matching points results in a jitter-free tracking and the offline learning prevents the tracking framework from drifting. We train the tracking system on the Cohn-Kanade database and analyze the dynamic of emotions and Action Units on the MMI database sequences. We perform accurate detection of facial expressions temporal segment and report experimental results