On-line Human Activity Recognition from Audio and Home Automation Sensors: comparison of sequential and non-sequential models in realistic Smart Homes

International audienceAutomatic human Activity Recognition (AR) is an important process for the provision of context-aware services in smart spaces such as voice-controlled smart homes. In this paper, we present an on-line Activities of Daily Living (ADL) recognition method for automatic identification within homes in which multiple sensors, actuators and automation equipment coexist, including audio sensors. Three sequence-based models are presented and compared: a Hidden Markov Model (HMM), Conditional Random Fields (CRF) and a sequential Markov Logic Network (MLN). These methods have been tested in two real Smart Homes thanks to experiments involving more than 30 participants. Their results were compared to those of three non-sequential models: a Support Vector Machine (SVM), a Random Forest (RF) and a non-sequential MLN. This comparative study shows that CRF gave the best results for on-line activity recognition from non-visual, audio and home automation sensors

User study and integration of assistive technologies for people with cognitive disabilities in their daily life activities

Journal of Ambient Intelligence and Smart Environments, vol. 7, no. 3, Gomez, Javier; Montoro, Germán, Evaluating Ambient Assisted Living Components and Systems, pp. 389-390, Copyright (2015), with permission from IOS PressThe present article summarizes the doctoral dissertation of Javier Gomez

Artificial Intelligence and Ambient Intelligence

This book includes a series of scientific papers published in the Special Issue on Artificial Intelligence and Ambient Intelligence at the journal Electronics MDPI. The book starts with an opinion paper on “Relations between Electronics, Artificial Intelligence and Information Society through Information Society Rules”, presenting relations between information society, electronics and artificial intelligence mainly through twenty-four IS laws. After that, the book continues with a series of technical papers that present applications of Artificial Intelligence and Ambient Intelligence in a variety of fields including affective computing, privacy and security in smart environments, and robotics. More specifically, the first part presents usage of Artificial Intelligence (AI) methods in combination with wearable devices (e.g., smartphones and wristbands) for recognizing human psychological states (e.g., emotions and cognitive load). The second part presents usage of AI methods in combination with laser sensors or Wi-Fi signals for improving security in smart buildings by identifying and counting the number of visitors. The last part presents usage of AI methods in robotics for improving robots’ ability for object gripping manipulation and perception. The language of the book is rather technical, thus the intended audience are scientists and researchers who have at least some basic knowledge in computer science

Customizing smart environments: a tabletop approach

Smart environments are becoming a reality in our society and the number of intelligent devices integrated in these spaces is in-creasing very rapidly. As the combination of intelligent elements will open a wide range of new opportunities to make our lives easier, final users should be provided with a simplified method of handling complex intelligent features. Specifying behavior in these environments can be difficult for non-experts, so that more efforts should be directed towards easing the customization tasks. This work presents an entirely visual rule editor based on dataflow expressions for interactive tabletops which allows be-havior to be specified in smart environments. An experiment was carried out aimed at evaluating the usability of the editor in terms of non-programmers understanding of the abstractions and concepts involved in the rule model, ease of use of the pro-posed visual interface and the suitability of the interaction mechanisms implemented in the editing tool. The study revealed that users with no previous programming experience were able to master the proposed rule model and editing tool for specifying be-havior in the context of a smart home, even though some minor usability issues were detected.We would like to thank all the volunteers that participated in the empirical study. Our thanks are also due to the ASIC/Polimedia team for their computer hardware support. This work was partially funded by the Spanish Ministry of Science and Innovation under the National R&D&I Program within the project CreateWorlds (TIN2010-20488). It also received support from a postdoctoral fellowship within the VALi+d Program of the Conselleria d'Educacio, Cultura I Esport (Generalitat Valenciana) awarded to Alejandro Catala (APOSTD/2013/013). The work of Patricia Pons has been supported by the Universitat Politecnica de Valencia under the "Beca de Excelencia" program, and currently by an FPU fellowship from the Spanish Ministry of Education, Culture and Sports (FPU13/03831).Pons Tomás, P.; Catalá Bolós, A.; Jaén Martínez, FJ. (2015). Customizing smart environments: a tabletop approach. Journal of Ambient Intelligence and Smart Environments. 7(4):511-533. https://doi.org/10.3233/AIS-150328S51153374[1]C. Becker, M. Handte, G. Schiele and K. Rothermel, PCOM – a component system for pervasive computing, in: Proc. of the Second IEEE International Conference on Pervasive Computing and Communications (PerCom’04), IEEE Computer Society, Washington, DC, USA, 2004, pp. 67–76.Bhatti, Z. W., Naqvi, N. Z., Ramakrishnan, A., Preuveneers, D., & Berbers, Y. (2014). Learning distributed deployment and configuration trade-offs for context-aware applications in Intelligent Environments. Journal of Ambient Intelligence and Smart Environments, 6(5), 541-559. doi:10.3233/ais-140274Bonino, D., & Corno, F. (2011). What would you ask to your home if it were intelligent? Exploring user expectations about next-generation homes. Journal of Ambient Intelligence and Smart Environments, 3(2), 111-126. doi:10.3233/ais-2011-0099[4]D. Bonino, F. Corno and L. Russis, A user-friendly interface for rules composition in intelligent environments, in: Ambient Intelligence – Software and Applications, Advances in Intelligent and Soft Computing, Vol. 92, Springer, Berlin, Heidelberg, 2011, pp. 213–217.[5]X. Carandang and J. Campbell, The design of a tangible user interface for a real-time strategy game, in: Proc. of the 34th International Conference on Information Systems (ICIS 2013), Association for Information Systems (AIS), 2013, pp. 3781–3790.Catalá, A., Garcia-Sanjuan, F., Jaen, J., & Mocholi, J. A. (2012). TangiWheel: A Widget for Manipulating Collections on Tabletop Displays Supporting Hybrid Input Modality. Journal of Computer Science and Technology, 27(4), 811-829. doi:10.1007/s11390-012-1266-4Catala, A., Pons, P., Jaen, J., Mocholi, J. A., & Navarro, E. (2013). A meta-model for dataflow-based rules in smart environments: Evaluating user comprehension and performance. Science of Computer Programming, 78(10), 1930-1950. doi:10.1016/j.scico.2012.06.010[8]C. Chen, Y. Xu, K. Li and S. Helal, Reactive programming optimizations in pervasive computing, in: Proc. of the 2010 10th IEEE/IPSJ International Symposium on Applications and the Internet (SAINT’10), IEEE Computer Society, Washington, DC, USA, 2010, pp. 96–104.Cook, D. J., Augusto, J. C., & Jakkula, V. R. (2009). Ambient intelligence: Technologies, applications, and opportunities. Pervasive and Mobile Computing, 5(4), 277-298. doi:10.1016/j.pmcj.2009.04.001Dey, A. K. (2009). Modeling and intelligibility in ambient environments. Journal of Ambient Intelligence and Smart Environments, 1(1), 57-62. doi:10.3233/ais-2009-0008[11]A.K. Dey, T. Sohn, S. Streng and J. Kodama, iCAP: Interactive prototyping of context-aware applications, in: Proc. of Pervasive Computing, Lecture Notes in Computer Science, Vol. 3968, Springer-Verlag, Berlin, Heidelberg, 2006, pp. 254–271.[12]N. Díaz, J. Lilius, M. Pegalajar and M. Delgado, Rapid prototyping of semantic applications in smart spaces with a visual rule language, in: Proc. of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, ACM, New York, NY, USA, 2013, pp. 1335–1338.Gámez, N., & Fuentes, L. (2011). FamiWare: a family of event-based middleware for ambient intelligence. Personal and Ubiquitous Computing, 15(4), 329-339. doi:10.1007/s00779-010-0354-0García-Herranz, M., Alamán, X., & Haya, P. A. (2010). Easing the Smart Home: A rule-based language and multi-agent structure for end user development in Intelligent Environments. Journal of Ambient Intelligence and Smart Environments, 2(4), 437-438. doi:10.3233/ais-2010-0085[17]J. Good, K. Howland and K. Nicholson, Young people’s descriptions of computational rules in role-playing games: An empirical study, in: Proc. of the 2010 IEEE Symposium on Visual Languages and Human-Centric Computing, IEEE, 2010, pp. 67–74.Gouin-Vallerand, C., Abdulrazak, B., Giroux, S., & Dey, A. K. (2013). A context-aware service provision system for smart environments based on the user interaction modalities. Journal of Ambient Intelligence and Smart Environments, 5(1), 47-64. doi:10.3233/ais-120190[19]S. Holloway and C. Julien, The case for end-user programming of ubiquitous computing environments, in: Proc. of the FSE/SDP Workshop on Future of Software Engineering Research (FoSER’10), ACM, New York, NY, USA, 2010, pp. 167–172.Horn, M. S., Crouser, R. J., & Bers, M. U. (2011). Tangible interaction and learning: the case for a hybrid approach. Personal and Ubiquitous Computing, 16(4), 379-389. doi:10.1007/s00779-011-0404-2[21]M.S. Horn, E.T. Solovey, R.J. Crouser and R.J.K. Jacob, Comparing the use of tangible and graphical programming languages for informal science education, in: Proc. of the SIGCHI Conference on Human Factors in Computing Systems (CHI’09), ACM, New York, NY, USA, 2009, pp. 975–984.Kelleher, C., & Pausch, R. (2005). Lowering the barriers to programming. ACM Computing Surveys, 37(2), 83-137. doi:10.1145/1089733.1089734[23]J. Lee, L. Garduño, E. Walker and W. Burleson, A tangible programming tool for creation of context-aware applications, in: Proc. of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp’13), ACM, New York, NY, USA, 2013, pp. 391–400.Lézoray, J.-B., Segarra, M.-T., Phung-Khac, A., Thépaut, A., Gilliot, J.-M., & Beugnard, A. (2011). A design process enabling adaptation in pervasive heterogeneous contexts. Personal and Ubiquitous Computing, 15(4), 353-363. doi:10.1007/s00779-010-0356-y[25]B.Y. Lim and A.K. Dey, Assessing demand for intelligibility in context-aware applications, in: Proc. of the 11th International Conference on Ubiquitous Computing (Ubicomp’09), ACM, New York, NY, USA, 2009, pp. 195–204.[26]B.Y. Lim and A.K. Dey, Evaluating Intelligibility usage and usefulness in a context-aware application, in: Proc. of the 15th International Conference on Human-Computer, Lecture Notes in Computer Science, Vol. 8008, Springer, Berlin, Heidelberg, 2013, pp. 92–101.[28]P. Marshall, Do tangible interfaces enhance learning? in: Proc. of the 1st International Conference on Tangible and Embedded Interaction (TEI’07), ACM, New York, NY, USA, 2007, pp. 163–170.[30]C. Maternaghan and K.J. Turner, A configurable telecare system, in: Proc. of the 4th International Conference on Pervasive Technologies Related to Assistive Environments (PETRA’11), ACM, New York, NY, USA, 2011, pp. 14:1–14:8.[31]D.A. Norman, The Invisible Computer, MITT Press, Cambridge, MA, USA, 1998.PANE, J. F., RATANAMAHATANA, C. «ANN», & MYERS, B. A. (2001). Studying the language and structure in non-programmers’ solutions to programming problems. International Journal of Human-Computer Studies, 54(2), 237-264. doi:10.1006/ijhc.2000.0410[33]P. Pons, A. Catala, J. Jaen and J.A. Mocholi, DafRule: Un modelo de reglas enriquecido mediante flujos de datos para la definición visual de comportamiento reactivo de entidades virtuales, in: Actas de las Jornadas de Ingeniería del Software y Bases de Datos (JISBD 2011), 2011, 989–1002.Rasch, K. (2014). An unsupervised recommender system for smart homes. Journal of Ambient Intelligence and Smart Environments, 6(1), 21-37. doi:10.3233/ais-130242[35]K. Ryall, C. Forlines, C. Shen and M.R. Morris, Exploring the effects of group size and table size on interactions with tabletop shared-display groupware, in: Proc. of the 2004 ACM Conference on Computer Supported Cooperative Work, ACM, New York, NY, USA, 2004, pp. 284–293.Schmidt, A. (2000). Implicit human computer interaction through context. Personal Technologies, 4(2-3), 191-199. doi:10.1007/bf01324126[37]S.D. Scott, M. Sheelagh, T. Carpendale and K.M. Inkpen, Territoriality in collaborative tabletop workspaces, in: Proc. of the 2004 ACM Conference on Computer Supported Cooperative Work, ACM, New York, NY, USA, 2004, pp. 294–303.Sapounidis, T., & Demetriadis, S. (2013). Tangible versus graphical user interfaces for robot programming: exploring cross-age children’s preferences. Personal and Ubiquitous Computing, 17(8), 1775-1786. doi:10.1007/s00779-013-0641-7Shadbolt, N. (2003). Brain power. IEEE Intelligent Systems, 18(3), 2-3. doi:10.1109/mis.2003.1200718Shafti, L. S., Haya, P. A., García-Herranz, M., & Pérez, E. (2013). Inferring ECA-based rules for ambient intelligence using evolutionary feature extraction. Journal of Ambient Intelligence and Smart Environments, 5(6), 563-587. doi:10.3233/ais-130232[41]A. Strawhacker, A. Sullivan and M.U. Bers, TUI, GUI, HUI: Is a bimodal interface truly worth the sum of its parts? in: Proc. of the 12th International Conference on Interaction Design and Children, ACM, New York, NY, USA, 2013, pp. 309–312.Sylla, C., Branco, P., Coutinho, C., & Coquet, E. (2011). TUIs vs. GUIs: comparing the learning potential with preschoolers. Personal and Ubiquitous Computing, 16(4), 421-432. doi:10.1007/s00779-011-0407-zTeruel, M. A., Navarro, E., López-Jaquero, V., Montero, F., Jaen, J., & González, P. (2012). Analyzing the understandability of Requirements Engineering languages for CSCW systems: A family of experiments. Information and Software Technology, 54(11), 1215-1228. doi:10.1016/j.infsof.2012.06.001[44]E. Tse, J. Histon, S.D. Scott and S. Greenberg, Avoiding interference: How people use spatial separation and partitioning in SDG workspaces, in: Proc. of the 2004 ACM Conference on Computer Supported Cooperative Work, ACM, New York, NY, USA, 2004, pp. 252–261.[45]P. Tuddenham, D. Kirk and S. Izadi, Graspables revisited: Multi-touch vs. tangible input for tabletop displays in acquisition and manipulation tasks, in: Proc. of the SIGCHI Conference on Human Factors in Computing Systems (CHI’10), ACM, New York, NY, USA, 2010, pp. 2223–2232.[46]A. Uribarren, J. Parra, R. Iglesias, J.P. Uribe and D. López de Ipiña, A middleware platform for application configuration, adaptation and interoperability, in: Proc. of the 2008 Second IEEE International Conference on Self-Adaptive and Self-Organizing Systems Workshops, IEEE Computer Society, Washington, DC, USA, 2008, pp. 162–167.Weiser, M. (1991). The Computer for the 21st Century. Scientific American, 265(3), 94-104. doi:10.1038/scientificamerican0991-94[48]C. Wohlin, P. Runeson, M. Höst, M.C. Ohlsson, B. Regnell and A. Wesslén, Experimentation in Software Engineering: An Introduction, 1st edn, Kluwer Academic Publishers, Norwell, MA, USA, 2000.Zuckerman, O., & Gal-Oz, A. (2013). To TUI or not to TUI: Evaluating performance and preference in tangible vs. graphical user interfaces. International Journal of Human-Computer Studies, 71(7-8), 803-820. doi:10.1016/j.ijhcs.2013.04.00

Preface to the 10th anniversary issue of the Journal on Ambient Intelligence and Smart Environments

The Editors in Chief of the JAISE journal reflect on the evolution of the technical area and the scientific community the publication has been serving for a decade

Designing a goal-oriented smart-home environment

The final publication is available at Springer via http://dx.doi.org/10.1007/s10796-016-9670-x[EN] Nowadays, systems are growing in power and in access to more resources and services. This situation makes it necessary to provide user-centered systems that act as intelligent assistants. These systems should be able to interact in a natural way with human users and the environment and also be able to take into account user goals and environment information and changes. In this paper, we present an architecture for the design and development of a goal-oriented, self-adaptive, smart-home environment. With this architecture, users are able to interact with the system by expressing their goals which are translated into a set of agent actions in a way that is transparent to the user. This is especially appropriate for environments where ambient intelligence and automatic control are integrated for the user’s welfare. In order to validate this proposal, we designed a prototype based on the proposed architecture for smart-home scenarios. We also performed a set of experiments that shows how the proposed architecture for human-agent interaction increases the number and quality of user goals achieved.This work is partially supported by the Spanish Government through the MINECO/FEDER project TIN2015-65515-C4-1-R.Palanca Cámara, J.; Del Val Noguera, E.; García-Fornes, A.; Billhard, H.; Corchado, JM.; Julian Inglada, VJ. (2016). Designing a goal-oriented smart-home environment. Information Systems Frontiers. 1-18. https://doi.org/10.1007/s10796-016-9670-xS118Alam, M. R., Reaz, M. B. I., & Ali, M. A. M. (2012). A review of smart homes: Past, present, and future. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 42(6), 1190–1203.Andrushevich, A., Staub, M., Kistler, R., & Klapproth, A. (2010). Towards semantic buildings: Goal-driven approach for building automation service allocation and control. In 2010 IEEE conference on emerging technologies and factory automation (ETFA) (pp. 1–6) IEEE.Ayala, I., Amor, M., & Fuentes, L. (2013). Self-configuring agents for ambient assisted living applications. Personal and Ubiquitous Computing, 17(6), 1159–1169.Cetina, C., Giner, P., Fons, J., & Pelechano, V. (2009). Autonomic computing through reuse of variability models at runtime: The case of smart homes. Computer, 42(10), 37–43.Cook, D. J. (2009). Multi-agent smart environments. Journal of Ambient Intelligence and Smart Environments, 1(1), 51–55.Dalpiaz, F., Giorgini, P., & Mylopoulos, J. (2009). An architecture for requirements-driven self-reconfiguration. In Advanced information systems engineering (pp. pp 246–260). Springer.De Silva, L. C., Morikawa, C., & Petra, I. M. (2012). State of the art of smart homes. Engineering Applications of Artificial Intelligence, 25(7), 1313–1321.Huhns, M., & et al. (2005). Research directions for service-oriented multiagent systems. IEEE Internet Computing, 9, 69–70.Iftikhar, M. U., & Weyns, D. (2014). Activforms: active formal models for self-adaptation. In SEAMS, (pp 125–134).Kucher, K., & Weyns, D. (2013). A self-adaptive software system to support elderly care. Modern Information Technology, MIT.Lieberman, H., & Espinosa, J. (2006). A goal-oriented interface to consumer electronics using planning and commonsense reasoning. In Proceedings of the 11th international conference on Intelligent user interfaces (pp. 226–233).Liu, H., & Singh, P. (2004). ConceptNet—a practical commonsense reasoning tool-kit. BT Technology Journal, 22(4), 211–226.Loseto, G., Scioscia, F., Ruta, M., & Di Sciascio, E. (2012). Semantic-based smart homes: a multi-agent approach. In 13th Workshop on objects and Agents (WOA 2012) (Vol. 892, pp. 49–55).Martin, D., Burstein, M., Hobbs, J., Lassila, O., McDermott, D., McIlraith, S., Narayanan, S., Paolucci, M., Parsia, B., Payne, T., & et al (2004). OWL-S: Semantic markup for web services. W3C Member Submission, 22, 2007–2004.Matthews, R. B., Gilbert, N. G., Roach, A., Polhill, J. G, & Gotts, N. M. (2007). Agent-based land-use models: a review of applications. Landscape Ecology, 22(10), 1447–1459.Molina, J. M., Corchado, J. M., & Bajo, J. (2008). Ubiquitous computing for mobile environments. In Issues in multi-agent systems (pp 33–57). Birkhäuser, Basel.Palanca, J., Navarro, M., Julian, V., & García-Fornes, A. (2012). Distributed goal-oriented computing. Journal of Systems and Software, 85(7), 1540–1557. doi: 10.1016/j.jss.2012.01.045 .Rao, A., & Georgeff, M. (1995). BDI agents: From theory to practice. In Proceedings of the first international conference on multi-agent systems (ICMAS95) (pp. 312–319).Reddy, Y. (2006). Pervasive computing: implications, opportunities and challenges for the society. In 1st International symposium on pervasive computing and applications (p. 5).de Silva, L., & Padgham, L. (2005). Planning as needed in BDI systems. International Conference on Automated Planning and Scheduling.Singh, P. (2002). The public acquisition of commonsense knowledge. In Proceedings of AAAI Spring symposium acquiring (and using) linguistic (and world) knowledge for information access