Semi-automated dialogue act classification for situated social agents in games

As a step toward simulating dynamic dialogue between agents and humans in virtual environments, we describe learning a model of social behavior composed of interleaved utterances and physical actions. In our model, utterances are abstracted as {speech act, propositional content, referent} triples. After training a classifier on 100 gameplay logs from The Restaurant Game annotated with dialogue act triples, we have automatically classified utterances in an additional 5,000 logs. A quantitative evaluation of statistical models learned from the gameplay logs demonstrates that semi-automatically classified dialogue acts yield significantly more predictive power than automatically clustered utterances, and serve as a better common currency for modeling interleaved actions and utterances

Extending MAM5 Meta-Model and JaCalIVE Framework to Integrate Smart Devices from Real Environments

[EN] This paper presents the extension of a meta-model (MAM5) and a framework based on the model (JaCalIVE) for developing intelligent virtual environments. The goal of this extension is to develop augmented mirror worlds that represent a real and virtual world coupled, so that the virtual world not only reflects the real one, but also complements it. A new component called a smart resource artifact, that enables modelling and developing devices to access the real physical world, and a human in the loop agent to place a human in the system have been included in the meta-model and framework. The proposed extension of MAM5 has been tested by simulating a light control system where agents can access both virtual and real sensor/actuators through the smart resources developed. The results show that the use of real environment interactive elements (smart resource artifacts) in agent-based simulations allows to minimize the error between simulated and real system.This work is partially supported by the TIN2009-13839-C03-01, TIN2011-27652-C03-01, 547CSD2007-00022, COST Action IC0801, FP7-294931 and the FPI grant AP2013-01276 548 awarded to Jaime-Andres Rincon.Rincón Arango, JA.; Poza Luján, JL.; Julian Inglada, VJ.; Posadas Yagüe, JL.; Carrascosa Casamayor, C. (2016). Extending MAM5 Meta-Model and JaCalIVE Framework to Integrate Smart Devices from Real Environments. PLoS ONE. 11(2):1-27. https://doi.org/10.1371/journal.pone.0149665S127112Luck, M., & Aylett, R. (2000). Applying artificial intelligence to virtual reality: Intelligent virtual environments. Applied Artificial Intelligence, 14(1), 3-32. doi:10.1080/088395100117142Barella A, Ricci A, Boissier O, Carrascosa C. MAM5: Multi-Agent Model For Intelligent Virtual Environments. In: 10th European Workshop on Multi-Agent Systems (EUMAS 2012); 2012. p. 16–30.Omicini, A., Ricci, A., & Viroli, M. (2008). Artifacts in the A&A meta-model for multi-agent systems. Autonomous Agents and Multi-Agent Systems, 17(3), 432-456. doi:10.1007/s10458-008-9053-xYu Ch, Nagpal R. Distributed Consensus and Self-Adapting Modular Robots. In: IROS-2008 workshop on Self-Reconfigurable Robots and Applications; 2008. Available from: http://www.isi.edu/robots/iros08wksp/Papers/iros08-wksp-paper.pdfLidoris G, Buss M. A Multi-Agent System Architecture for Modular Robotic Mobility Aids. In: European Robotics Symposium 2006; 2006. p. 15–26. Available from: http://link.springer.com/chapter/10.1007/11681120_2Yu, C.-H., & Nagpal, R. (2010). A Self-adaptive Framework for Modular Robots in a Dynamic Environment: Theory and Applications. The International Journal of Robotics Research, 30(8), 1015-1036. doi:10.1177/0278364910384753Barbero A, González-Rodríguez MS, de Lara J, Alfonseca M. Multi-Agent Simulation of an Educational Collaborative Web System. In: European Simulation and Modelling Conference; 2007. Available from: http://sistemas-humano-computacionais.wikidot.com/local--files/capitulo:colaboracao-auxiliada-por-computador/%5BBarbero%202007%5D%20Multi-Agent%20Simulation%20of%20an%20Educational%20Collaborative%20Web%20System.pdfRanathunga S, Cranefield S, Purvis MK. Interfacing a cognitive agent platform with a virtual world: a case study using Second Life. In: AAMAS; 2011. p. 1181–1182. Available from: http://www.aamas-conference.org/Proceedings/aamas2011/papers/B20.pdfAndreoli R, De Chiara R, Erra U, Scarano V. Interactive 3d environments by using videogame engines. In: Information Visualisation, 2005. Proceedings. Ninth International Conference on. IEEE; 2005. p. 515–520. Available from: http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1509124Dignum, F. (2011). Agents for games and simulations. Autonomous Agents and Multi-Agent Systems, 24(2), 217-220. doi:10.1007/s10458-011-9169-2dos Santos C, Osorio F. AdapTIVE: an intelligent virtual environment and its application in e-commerce. In: Computer Software and Applications Conference, 2004. COMPSAC 2004. Proceedings of the 28th Annual International; 2004. p. 468–473 vol.1.Kazemi, A., Fazel Zarandi, M. H., & Moattar Husseini, S. M. (2008). A multi-agent system to solve the production–distribution planning problem for a supply chain: a genetic algorithm approach. The International Journal of Advanced Manufacturing Technology, 44(1-2), 180-193. doi:10.1007/s00170-008-1826-5Dimuro GP, Costa ACdR, Gonçalves LV, Hubner A. Interval-valued Hidden Markov Models for recognizing personality traits in social exchanges in open multiagent systems. Repositório Institucional da Universidade Federal do Rio Grande. 2008;.Woźniak, M., Graña, M., & Corchado, E. (2014). A survey of multiple classifier systems as hybrid systems. Information Fusion, 16, 3-17. doi:10.1016/j.inffus.2013.04.006Jia L, Zhenjiang M. Entertainment Oriented Intelligent Virtual Environment with Agent and Neural Networks. In: IEEE International Workshop on Haptic, Audio and Visual Environments and Games, 2007. HAVE 2007; 2007. p. 90–95.Corchado, E., Woźniak, M., Abraham, A., de Carvalho, A. C. P. L. F., & Snášel, V. (2014). Recent trends in intelligent data analysis. Neurocomputing, 126, 1-2. doi:10.1016/j.neucom.2013.07.001Ricci A, Viroli M, Omicini A. Give agents their artifacts: the A&A approach for engineering working environments in MAS. In: Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems; 2007. p. 150. Available from: http://dl.acm.org/citation.cfm?id=1329308Barella, A., Valero, S., & Carrascosa, C. (2009). JGOMAS: New Approach to AI Teaching. IEEE Transactions on Education, 52(2), 228-235. doi:10.1109/te.2008.925764Behrens, T. M., Hindriks, K. V., & Dix, J. (2010). Towards an environment interface standard for agent platforms. Annals of Mathematics and Artificial Intelligence, 61(4), 261-295. doi:10.1007/s10472-010-9215-9Ricci A, Viroli M, Omicini A. A general purpose programming model & technology for developing working environments in MAS. In: 5th International Workshop Programming Multi-Agent Systems(PROMAS 2007); 2007. p. 54–69. Available from: http://lia.deis.unibo.it/~ao/pubs/pdf/2007/promas.pdfChee-Yee Chong, & Kumar, S. P. (2003). Sensor networks: Evolution, opportunities, and challenges. Proceedings of the IEEE, 91(8), 1247-1256. doi:10.1109/jproc.2003.814918Kushner D. The making of arduino. IEEE Spectrum. 2011;26.Schmidt, A., & van Laerhoven, K. (2001). How to build smart appliances? IEEE Personal Communications, 8(4), 66-71. doi:10.1109/98.944006Salzmann C, Gillet D. Smart device paradigm standardization for online labs. In: 4th IEEE Global Engineering Education Conference (EDUCON); 2013.Gonzalez-Jorge, H., Riveiro, B., Vazquez-Fernandez, E., Martínez-Sánchez, J., & Arias, P. (2013). Metrological evaluation of Microsoft Kinect and Asus Xtion sensors. Measurement, 46(6), 1800-1806. doi:10.1016/j.measurement.2013.01.011Cook, D. J., & Das, S. K. (2007). How smart are our environments? An updated look at the state of the art. Pervasive and Mobile Computing, 3(2), 53-73. doi:10.1016/j.pmcj.2006.12.001Compton, M., Barnaghi, P., Bermudez, L., García-Castro, R., Corcho, O., Cox, S., … Taylor, K. (2012). The SSN ontology of the W3C semantic sensor network incubator group. Journal of Web Semantics, 17, 25-32. doi:10.1016/j.websem.2012.05.003Munera, E., Poza-Lujan, J.-L., Posadas-Yagüe, J.-L., Simó-Ten, J.-E., & Noguera, J. (2015). Dynamic Reconfiguration of a RGBD Sensor Based on QoS and QoC Requirements in Distributed Systems. Sensors, 15(8), 18080-18101. doi:10.3390/s150818080Castrillón-Santan, M., Lorenzo-Navarro, J., & Hernández-Sosa, D. (2014). Conteo de personas con un sensor RGBD comercial. Revista Iberoamericana de Automática e Informática Industrial RIAI, 11(3), 348-357. doi:10.1016/j.riai.2014.05.006Rincon JA, Julian V, Carrascosa C. An Emotional-based Hybrid Application for Human-Agent Societies. In: 10th International Conference on Soft Computing Models in Industrial and Environmental Applications. vol. 368; 2015. p. 203–214.Rincon JA, Julian V, Carrascosa C. Applying a Social Emotional Model in Human-Agent Societies. In: Workshop WIHAS’15. Intelligent Human-Agent Societies‥ vol. 524 of CCIS; 2015. p. 377–388.Leccese, F., Cagnetti, M., & Trinca, D. (2014). A Smart City Application: A Fully Controlled Street Lighting Isle Based on Raspberry-Pi Card, a ZigBee Sensor Network and WiMAX. Sensors, 14(12), 24408-24424. doi:10.3390/s141224408Mateevitsi V, Haggadone B, Leigh J, Kunzer B, Kenyon RV. Sensing the environment through SpiderSense. In: Proceedings of the 4th Augmented Human International Conference. ACM; 2013. p. 51–57.Kavitha R, Thiyagarajan N. Distributed Intelligent Street Lamp Monitoring and Control System Based on Zigbee. International Journal of Science and Research (IJSR) PP; p. 2319–7064.Pan, M.-S., Yeh, L.-W., Chen, Y.-A., Lin, Y.-H., & Tseng, Y.-C. (2008). A WSN-Based Intelligent Light Control System Considering User Activities and Profiles. IEEE Sensors Journal, 8(10), 1710-1721. doi:10.1109/jsen.2008.2004294Villarrubia, G., De Paz, J., Bajo, J., & Corchado, J. (2014). Ambient Agents: Embedded Agents for Remote Control and Monitoring Using the PANGEA Platform. Sensors, 14(8), 13955-13979. doi:10.3390/s14081395

Simulated role-playing from crowdsourced data

Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 173-178).Collective Artificial Intelligence (CAl) simulates human intelligence from data contributed by many humans, mined for inter-related patterns. This thesis applies CAI to social role-playing, introducing an end-to-end process for compositing recorded performances from thousands of humans, and simulating open-ended interaction from this data. The CAI process combines crowdsourcing, pattern discovery, and case-based planning. Content creation is crowdsourced by recording role-players online. Browser-based tools allow nonexperts to annotate data, organizing content into a hierarchical narrative structure. Patterns discovered from data power a novel system combining plan recognition with case-based planning. The combination of this process and structure produces a new medium, which exploits a massive corpus to realize characters who interact and converse with humans. This medium enables new experiences in videogames, and new classes of training simulations, therapeutic applications, and social robots. While advances in graphics support incredible freedom to interact physically in simulations, current approaches to development restrict simulated social interaction to hand-crafted branches that do not scale to the thousands of possible patterns of actions and utterances observed in actual human interaction. There is a tension between freedom and system comprehension due to two bottlenecks, making open-ended social interaction a challenge. First is the authorial effort entailed to cover all possible inputs. Second, like other cognitive processes, imagination is a bounded resource. Any individual author only has so much imagination. The convergence of advances in connectivity, storage, and processing power is bringing people together in ways never before possible, amplifying the imagination of individuals by harnessing the creativity and productivity of the crowd, revolutionizing how we create media, and what media we can create. By embracing data-driven approaches, and capitalizing on the creativity of the crowd, authoring bottlenecks can be overcome, taking a step toward realizing a medium that robustly supports player choice. Doing so requires rethinking both technology and division of labor in media production. As a proof of concept, a CAI system has been evaluated by recording over 10,000 performances in The Restaurant Game, automating an Al-controlled waitress who interacts in the world, and converses with a human via text or speech. Quantitative results demonstrate how CAI supports significantly more open-ended interaction with humans, while focus groups reveal factors for improving engagement.by Jeffrey David Orkin.Ph.D

Niños que videojuegan, videojuegos que estructuran tiempos : cognición en los bordes del tiempo irreversible.

Este estudio examina cómo se manifiestan y despliegan diversas formas de cognición distribuida y corporalizada en el curso de la actividad de videojuego. Supone que cuando la persona no puede hacerse a una comprensión lógica anticipada de la tarea y cuando se imponen restricciones de tiempo para realizar esa tarea, necesariamente apela a un conjunto de procedimientos adaptativos –particularmente creativos- orientados a resolver aquello que, de manera puramente lógica, no puede resolverse. Ruidosa, corporalmente inestable, emocionalmente exuberante, la cognición puesta en situaciones límites, apela a toda clase de recursos corporales para atender aquellas tareas no anticipables de manera lógica y puramente mental. Aunque abundan las referencias acerca de la importancia de los abordajes enactivos y las derivas corporalizadas de la cognición a la hora de comprender qué pasa cuando los niños videojuegan, los estudios empíricos han sido menos frecuentes y detallados. “Niños que videojuegan, videojuegos que estructuran tiempos” es una investigación doctoral que examina el comportamiento corporal, elocutivo y emocional de los niños mientras videojuegan, y pone el énfasis en que este entramado corporalizado se configura de manera diferenciada según tipos de videojuegos, según se gana en pericia y dependiendo del estatuto de los eventos del mundo del videojuego en relación con la actividad del videojugador. Para poder comprender la dimensión corporalizada de la práctica de videojuego hace falta poner al centro el hecho de que se despliega en el tiempo irreversible, como un sistema abierto y dinámico, en torno a los eventos del mundo del videojuego. El estudio cifra en la situación, en el carácter situado de la práctica de videojuego, todo su empeño: cree que más allá de la disputa académica en torno al énfasis en la estructura y dimensiones expresivas del videojuego, y las reglas, formas de incentivo y castigo del videojuego, el centro de los videojuegos –en tanto práctica social- está en la ejecución, en esta forma particular de ensamblaje agente humano-agente no humano que es el videojugar, desplegándose en el tiempo