Ontology-based data semantic management and application in IoT- and cloud-enabled smart homes

The application of emerging technologies of Internet of Things (IoT) and cloud computing have increasing the popularity of smart homes, along with which, large volumes of heterogeneous data have been generating by home entities. The representation, management and application of the continuously increasing amounts of heterogeneous data in the smart home data space have been critical challenges to the further development of smart home industry. To this end, a scheme for ontology-based data semantic management and application is proposed in this paper. Based on a smart home system model abstracted from the perspective of implementing users’ household operations, a general domain ontology model is designed by defining the correlative concepts, and a logical data semantic fusion model is designed accordingly. Subsequently, to achieve high-efficiency ontology data query and update in the implementation of the data semantic fusion model, a relational-database-based ontology data decomposition storage method is developed by thoroughly investigating existing storage modes, and the performance is demonstrated using a group of elaborated ontology data query and update operations. Comprehensively utilizing the stated achievements, ontology-based semantic reasoning with a specially designed semantic matching rule is studied as well in this work in an attempt to provide accurate and personalized home services, and the efficiency is demonstrated through experiments conducted on the developed testing system for user behavior reasoning

Training the Behaviour Preferences on Context Changes

Personalized ambient intelligent systems should meet changes in user’s needs, which evolve over time. Our objective is to create an adaptive system that learns the user behaviour preferences. We propose *BAM – * Behaviour Adaptation Mechanism, a neural-network based control system that is trained, supervised by user’s (affective) feedback in real-time. The system deduces the preferred behaviour, based on the detection of affective state’s valence (negative, neutral and positive) from facial features analysis. The neural network is retrained periodically with the updated training set, obtained from the interpretation of the user’s reaction to the system’s decisions. We investigated how many training examples, rendered from user’s behaviour, are required in order to train the neural network so that it reaches an accuracy of at least 75%. We present the evolution of behaviour preference learning parameters when the number of context elements increases

A User Profile Ontology For Situation-Aware Social Networking

International audienceToday, more and more people possess mobile devices. This enables them to have access to a wide range of services, but also to be contacted anytime, anywhere, which can cause discom- fort. Users should have full control on who can reach them and how, depending on their current situation: when at work, we wish a friend didn't interrupted us, but when having a family dinner, we wish a call related to work didn't occur. Even more, situation changes need to be detected in real-time, since preferences change a lot. We present in this paper an ontology-based user profile model, that allows users to have a situation-aware social network, by controlling how reachable they are for specific categories of people in a given situation

Addressing the evolution of automated user behaviour patterns by runtime model interpretation

The final publication is available at Springer via http://dx.doi.org/10.1007/s10270-013-0371-3The use of high-level abstraction models can facilitate and improve not only system development but also runtime system evolution. This is the idea of this work, in which behavioural models created at design time are also used at runtime to evolve system behaviour. These behavioural models describe the routine tasks that users want to be automated by the system. However, users¿ needs may change after system deployment, and the routine tasks automated by the system must evolve to adapt to these changes. To facilitate this evolution, the automation of the specified routine tasks is achieved by directly interpreting the models at runtime. This turns models into the primary means to understand and interact with the system behaviour associated with the routine tasks as well as to execute and modify it. Thus, we provide tools to allow the adaptation of this behaviour by modifying the models at runtime. This means that the system behaviour evolution is performed by using high-level abstractions and avoiding the costs and risks associated with shutting down and restarting the system.This work has been developed with the support of MICINN, under the project EVERYWARE TIN2010-18011, and the support of the Christian Doppler Forschungsgesellschaft and the BMWFJ, Austria.Serral Asensio, E.; Valderas Aranda, PJ.; Pelechano Ferragud, V. (2013). Addressing the evolution of automated user behaviour patterns by runtime model interpretation. Software and Systems Modeling. https://doi.org/10.1007/s10270-013-0371-3SWeiser, M.: The computer of the 21st century. Sci. Am. 265, 66–75 (1991)Serral, E., Valderas, P., Pelechano, V.: Context-adaptive coordination of pervasive services by interpreting models during runtime. Comput. J. 56(1), 87–114 (2013)Ajila, S.A., Alam, S.: Using a formal language constructs for software model evolution. In: Third IEEE International Conference on Semantic Computing (IEEE-ICSC 2009). Berkeley, CA, USA, pp. 390–395 (2009)Bennett, K., Rajlich, V.: Software Maintenance and Evolution: A Roadmap. In: 22nd International Conference on Software Engineering (ICSE 2000). Limerick, Ireland, pp. 75–87 (2000)Mens, T.: The ERCIM working group on software evolution: the past and the future. In: Proceedings of the Joint International and Annual ERCIM Workshops on Principles of Software Evolution (IWPSE) and Software Evolution (Evol) Workshops. ACM (2009)Mens, T., Wermelinger, M., Ducasse, S., Demeyer, S., Hirschfeld, R.: Challenges in software evolution. In: Report of the ChaSE 2005 Workshop Organised by the ERCIM Working Group on Software Evolution. IWPSE-05. Lisbon, Portugal, pp. 13–22 (2005)Hirschfeld, R., Kawamura, K., Berndt, H.: Dynamic service adaptation for runtime system extensions. In: Wireless On-Demand Network Systems, pp. 227–240. Springer, Berlin, Heidelberg, Madonna di Campiglio, Italy (2004)Lientz, B.P., Swanson, E.B.: Software maintenance management: a study of the maintenance of computer applications software in 487 data processing organizations. Addison-Wesley, Reading, MA (1980)Buckley, J., Mens, T., Zenger, M., Rashid, A., Kniesel, G.: Towards a taxonomy of software change. J. Softw. Maint. Evolut. Res. Pract. 17(5), 309–332 (2003)Hardian, B., Indulska, J., Henricksen, K.: Balancing autonomy and user control in context-aware systems—a survey. In: CoMoRea, IEEE PerCom Workshops 2006. (2006)Biegel, G., Cahill, V.: A framework for developing mobile, context-aware applications. In: The 2nd IEEE Conference on Pervasive Computing and Communication (PerCom), pp. 361–365 (2004)Hofer, T., Schwinger, W., Pichler, M., Leonhartsberger, G., Altmann, J.: Context-awareness on mobile devices—the hydrogen approach. In: The 36th Annual Hawaii International Conference on System Sciences, pp. 292–302 (2002)Dey, A.K.: Understanding and using context. Pers. Ubiquitous Comput. 5(1), 4–7 (2001)Sheng, Q.Z., Benatallah, B.: ContextUML: a UML-based modelling language for model-driven development of context-aware web services. In: Proceedings of the International Conference on Mobile, Business (ICMB’05). pp. 206–212 (2005)Henricksen, K., Indulska, J.: A software engineering framework for context-aware pervasive computing. In: Proceedings of the Second IEEE Annual Conference on Pervasive Computing and Communications (PerCom 2004), pp. 77–86. IEEE, Orlando, FL, USA (2004)Baldauf, M., Dustdar, S., Rosenberg, F.: A survey on context-aware systems. Int. J. Ad Hoc Ubiquitous Comput. 2(4), 263–277 (2007)Ye, J., Coyle, L., Dobson, S., Nixon, P.: Ontology-based models in pervasive computing systems. Knowl. Eng. Rev. 22(4), 315–347 (2007)Chen, H., Finin, T., Joshi, A.: An ontology for context-aware pervasive computing environments. Special Issue on Ontologies for Distributed Systems. Knowl. Eng. Rev. 18(3), 197–207 (2004)Welty, C., McGuinness, D.L.: OWL Web Ontology Language Guide. vol. W3C Recomm. W3C Recommendation 10 Feb 2004 (2004)Shepherd, A.: HTA as a framework for task analysis. Ergonomics 41, 1537–1552 (1998)Serral, E., Valderas, P., Pelechano, V.: Towards the model driven development of context-aware pervasive systems. Special Issue on Context Modelling, Reasoning and Management. PMC 6(2), 254–280 (2010)Serral, E.: Automating Routine Tasks in Smart Environments. A Context-aware Model-driven Approach, Technical University of Valencia (2011)Mellor, S.J., Balcer, M.J.: Executable UML: A Foundation for Model Driven Architecture. Addison-Wesley, Indianapolis (2002)Muñoz, J., Ferragud, D.V.P.: Model Driven Development of Pervasive Systems. Building a Software Factory. Universidad Politécnica de Valencia, Valencia (2008)Juric, M.B., Sarang, P.: Business Process Execution Language for Web Services: BPEL and BPEL4WS (2006)Loke, S.W., Smanchat, S., Ling, S., Indrawan, M.: Formal mirror models: an approach to just-in-time reasoning for device ecologies. Int. J. Smart Home 2(1), 15–32 (2008)Code Generation conference. http://www.codegeneration.net/cg2010/ (2010)Guy, M.: Report 2: API Good Practice Good practice for provision of and consuming APIs. UKOLN (2009)Bloch, J.: How to design a good API and why it matters. pp. 506–507 (2005)Sirin, E., Parsia, B., Grau, B.C., Kalyanpur, A., Katz, Y.: Pellet: A practical OWL-DL reasoner. J. Web Semant. 5(2), 51–53 (2007)Bernstein, P.: Multiversion concurrency control—theory and algorithms. ACM Trans. Database Syst. 8(4), 465–484 (1983)Cooper, S., Dann, W., Pausch, R.: Alice: a 3-D tool for introductory programming concepts. J. Comput. Sci. Coll. 15, 107–116 (2000)Pérez, F., Valderas, P.: Allowing end-users to actively participate within the elicitation of pervasive system requirements through immediate visualization. In: Fourth International Workshop on Requirements Engineering Visualization (REV), pp. 31–40. IEEE, Atlanta, Georgia, USA (2009)Lieberman, H., Paternó, F., Wulf, V.: End User Development. Springer, Dordrecht (2006)Nielsen, J.: Usability Engineering. Morgan Kaufmann Publishers Inc, San Francisco (1993)Van Welie, M., Trætteberg, H.: Interaction Patterns in User, Interfaces. pp. 13–16 (2000)Galitz, W.O.: The Essential Guide to User Interface Design: An Introduction to GUI Design Principles and Techniques. Wiley, New York (2002)Kitchenham, B., Pickard, L., Pfleeger, S.L.: Case studies for method and tool evaluation. Softw. IEEE 12(4), 52–62 (1995)Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering. Springer, Berlin (2012)Jones, J.V.: Applied software measurement: assuring productivity & quality (2nd ed’97). McGraw-Hill, New York (1997)Strang, T., Linnhoff-Popien, C.: A context modeling survey. In: First International Workshop on Advanced Context Modelling, Reasoning And Management at UbiComp (2004)Lewis, J.R.: Psychometric Evaluation of an After-Scenario Questionnaire for Computer Usability Studies? The ASQ. SIGCHI Bulletin (1991)Cook, D.J., Youngblood, M., Heierman, I.I.I.E.O., Gopalratnam, K., Rao, S., Litvin, A., Khawaja, F.: MavHome: An Agent-based Smart Home. In: First IEEE International Conference on Pervasive Computing and, Communications (PerCom’03), pp. 521–524 (2003)Hagras, H., Callaghan, V., Colley, M., Clarke, G., Pounds-Cornish, A., Duman, H.: Creating an ambient-intelligence environment using embedded agents. IEEE Intell. Syst. 19(6), 12–20 (2004)Rashidi, P., Cook, D.J.: Keeping the resident in the loop: adapting the smart home to the user. IEEE Trans. Syst. Man Cybern. 39(5), 949–959 (2009)Webb, G.I., Pazzani, M.J., Billsus, D.: Machine learning for user modeling. User model. User-Adapt Interact. 11(1–2), 19–29 (2001)Valiant, L.G.: A theory of the learnable. Commun. ACM 27(11), 1134–1142 (1984)Serral, E., Valderas, P., Pelechano, V.: (2011) Improving the cold-start problem in user task automation by using models at runtime. In: Information Systems Development, pp. 671–683. (2011)García-Herranz, M., Haya, P.A., Esquivel, A., Montoro, G., Alamán, X.: Easing the smart home: semi-automatic adaptation in perceptive environments. J. Univers. Comput. Sci. 14(9), 1529–1544 (2008)Henricksen, K., Indulska, J., Rakotonirainy, A.: Using context and preferences to implement self-adapting pervasive computing applications. Sofw. Pract. Exp. 36(11–12), 1307–1330 (2006)Johnson, P.: Tasks and situations: considerations for models and design principles in human computer interaction, pp. 1199–1204. HCI International. Munich, Germany (1999)Cook, D.J., Das, S.K.: Smart environments: technologies, protocols, and applications, vol. 43. Wiley-Interscience, New York (2005)Paternò, F.: ConcurTaskTrees: an Engineered approach to model-based design of interactive systems. In: The Handbook of Analysis for Human-Computer Interaction, pp. 483–500 (2002)Pribeanu, C., Limbourg, Q., Vanderdonckt1, J.: Task modelling for context-sensitive user interfaces. In: Interactive Systems: Design, Specification, and Verification (DSV-IS), pp. 49–68. Springer, Berlin, Heidelberg 2001, Glasgow, Scotland, UK (2001)Souchon, N., Limbourg, Q., Vanderdonckt., J.: Task modelling in multiple contexts of use. In: Interactive Systems: Design, Specification, and Verification (DSV-IS), pp. 59–73 (2002)Huang, R., Cao, Q., Zhou, J., Sun, D., Su, Q.: Context-aware active task discovery for pervasive computing. In: International Conference on Computer Science and Software Engineering, pp. 463–466. IEEE, Wuhan, China (2008)Sousa, J.P., Poladian, V., Garlan, D., Schmerl, B.: Task-based adaptation for ubiquitous computing. IEEE Trans. Syst. Man Cybern. 36(3), 328–340 (2006)Masuoka, R., Parsia, B., Labrou, Y.: Task Computing—The Semantic Web Meets Pervasive Computing. In: 2nd International Semantic Web Conference on the Semantic Web (ISWC 2003), pp. 866–881. vol. LNCS 2870. Sanibel Island, FL, USA (2003)Oreizy, P., Gorlick, M.M., Taylor, R.N., Heimbigner, D., Johnson, G., Medvidovic, N., Quilici, A., Rosenblum, D.S., Wolf, A.L.: An architecture-based approach to self-adaptive software. IEEE Intell. Syst. Their Appl. 14(3), 54–62 (1999)Floch, J., Hallsteinsen, S., Stav, E., Eliassen, F., Lund, K., Gjørven, E.: Using Architecture Models for Runtime Adaptability. IEEE Software. 23(2), 62–70 (2006)Morin, B., Jézéquel, J.-M., Fleurey, F., Solberg, A.: Models at runtime to support dynamic adaptation. IEEE Comput. Soc. pp. 46–53 (2009)Cetina, C., Giner, P., Fons, J., Pelechano, V.: Using feature models for developing self-configuring smart homes. In: Fifth International Conference on Autonomic and Autonomous Systems, pp. 179–188. IEEE, Valencia, Spain (2009)Garlan, D., Schmerl, B.: Using architectural models at runtime: research challenges. In: Proceedings of the European Workshop on Software Architectures, pp. 200–205. Springer, Berlin, Heidelberg, St Andrews, UK (2004)Blumendorf, M., Lehmann, G., Feuerstack, S., Albayrak, S.: Executable models for human-computer interaction. In: Interactive Systems, Design, Specification, and Verification Workshop (DSV-IS 2008), pp. 238–251. Springer Berlin Heidelberg, Kingston, Canada (2008)Ballagny, C., Hameurlain, N., Barbier, F.: MOCAS: a state-based component model for self-adaptation. In: Third IEEE International Conference on Self-Adaptive and Self-Organizing Systems, pp. 206–215. IEEE, San Francisco, California (2009)Amoui, M., Derakhshanmanesh, M., Ebert, J., Tahvildari, L.: Achieving dynamic adaptation via management and interpretation of runtime models. J. Syst. Softw. 85(12), 2720–2737 (2012)Blair, G., Bencomo, N., France, R.B.: [email protected]. IEEE Comput. 42, 22–27 (2009)Zhang, J., Cheng, B.H.C.: Model based development of dynamically adaptive software. In: International Conference on Software Engineering (ICSE’06), pp. 371–380. ACM, Shanghai, China (2006

An information assistant system for the prevention of tunnel vision in crisis management

In the crisis management environment, tunnel vision is a set of bias in decision makers’ cognitive process which often leads to incorrect understanding of the real crisis situation, biased perception of information, and improper decisions. The tunnel vision phenomenon is a consequence of both the challenges in the task and the natural limitation in a human being’s cognitive process. An information assistant system is proposed with the purpose of preventing tunnel vision. The system serves as a platform for monitoring the on-going crisis event. All information goes through the system before arrives at the user. The system enhances the data quality, reduces the data quantity and presents the crisis information in a manner that prevents or repairs the user’s cognitive overload. While working with such a system, the users (crisis managers) are expected to be more likely to stay aware of the actual situation, stay open minded to possibilities, and make proper decisions

A knowledge-based approach towards human activity recognition in smart environments

For many years it is known that the population of older persons is on the rise. A recent report estimates that globally, the share of the population aged 65 years or over is expected to increase from 9.3 percent in 2020 to around 16.0 percent in 2050 [1]. This point has been one of the main sources of motivation for active research in the domain of human activity recognition in smart-homes. The ability to perform ADL without assistance from other people can be considered as a reference for the estimation of the independent living level of the older person. Conventionally, this has been assessed by health-care domain experts via a qualitative evaluation of the ADL. Since this evaluation is qualitative, it can vary based on the person being monitored and the caregiver\u2019s experience. A significant amount of research work is implicitly or explicitly aimed at augmenting the health-care domain expert\u2019s qualitative evaluation with quantitative data or knowledge obtained from HAR. From a medical perspective, there is a lack of evidence about the technology readiness level of smart home architectures supporting older persons by recognizing ADL [2]. We hypothesize that this may be due to a lack of effective collaboration between smart-home researchers/developers and health-care domain experts, especially when considering HAR. We foresee an increase in HAR systems being developed in close collaboration with caregivers and geriatricians to support their qualitative evaluation of ADL with explainable quantitative outcomes of the HAR systems. This has been a motivation for the work in this thesis. The recognition of human activities \u2013 in particular ADL \u2013 may not only be limited to support the health and well-being of older people. It can be relevant to home users in general. For instance, HAR could support digital assistants or companion robots to provide contextually relevant and proactive support to the home users, whether young adults or old. This has also been a motivation for the work in this thesis. Given our motivations, namely, (i) facilitation of iterative development and ease in collaboration between HAR system researchers/developers and health-care domain experts in ADL, and (ii) robust HAR that can support digital assistants or companion robots. There is a need for the development of a HAR framework that at its core is modular and flexible to facilitate an iterative development process [3], which is an integral part of collaborative work that involves develop-test-improve phases. At the same time, the framework should be intelligible for the sake of enriched collaboration with health-care domain experts. Furthermore, it should be scalable, online, and accurate for having robust HAR, which can enable many smart-home applications. The goal of this thesis is to design and evaluate such a framework. This thesis contributes to the domain of HAR in smart-homes. Particularly the contribution can be divided into three parts. The first contribution is Arianna+, a framework to develop networks of ontologies - for knowledge representation and reasoning - that enables smart homes to perform human activity recognition online. The second contribution is OWLOOP, an API that supports the development of HAR system architectures based on Arianna+. It enables the usage of Ontology Web Language (OWL) by the means of Object-Oriented Programming (OOP). The third contribution is the evaluation and exploitation of Arianna+ using OWLOOP API. The exploitation of Arianna+ using OWLOOP API has resulted in four HAR system implementations. The evaluations and results of these HAR systems emphasize the novelty of Arianna+

Ontology for autonomous robotics

Creating a standard for knowledge representation and reasoning in autonomous robotics is an urgent task if we consider recent advances in robotics as well as predictions about the insertion of robots in human daily life. Indeed, this will impact the way information is exchanged between multiple robots or between robots and humans and how they can all understand it without ambiguity. Indeed, Human Robot Interaction (HRI) represents the interaction of at least two cognition models (Human and Robot). Such interaction informs task composition, task assignment, communication, cooperation and coordination in a dynamic environment, requiring a flexible representation. Hence, this paper presents the IEEE RAS Autonomous Robotics (AuR) Study Group, which is a spin-off of the IEEE Ontologies for Robotics and Automation (ORA) Working Group, and its ongoing work to develop the first IEEE-RAS ontology standard for autonomous robotics. In particular, this paper reports on the current version of the ontology for autonomous robotics as well as on its first implementation successfully validated for a human-robot interaction scenario, demonstrating the developed ontology’s strengths which include semantic interoperability and capability to relate ontologies from different fields for knowledge sharing and interactions.info:eu-repo/semantics/publishedVersio

A hybrid approach to recognising activities of daily living from object use in the home environment

Accurate recognition of Activities of Daily Living (ADL) plays an important role in providing assistance and support to the elderly and cognitively impaired. Current knowledge-driven and ontology-based techniques model object concepts from assumptions and everyday common knowledge of object use for routine activities. Modelling activities from such information can lead to incorrect recognition of particular routine activities resulting in possible failure to detect abnormal activity trends. In cases where such prior knowledge are not available, such techniques become virtually unemployable. A significant step in the recognition of activities is the accurate discovery of the object usage for specific routine activities. This paper presents a hybrid framework for automatic consumption of sensor data and associating object usage to routine activities using Latent Dirichlet Allocation (LDA) topic modelling. This process enables the recognition of simple activities of daily living from object usage and interactions in the home environment. The evaluation of the proposed framework on the Kasteren and Ordonez datasets show that it yields better results compared to existing techniques

Automatic Generation of Personalized Recommendations in eCoaching

Denne avhandlingen omhandler eCoaching for personlig livsstilsstøtte i sanntid ved bruk av informasjons- og kommunikasjonsteknologi. Utfordringen er å designe, utvikle og teknisk evaluere en prototyp av en intelligent eCoach som automatisk genererer personlige og evidensbaserte anbefalinger til en bedre livsstil. Den utviklede løsningen er fokusert på forbedring av fysisk aktivitet. Prototypen bruker bærbare medisinske aktivitetssensorer. De innsamlede data blir semantisk representert og kunstig intelligente algoritmer genererer automatisk meningsfulle, personlige og kontekstbaserte anbefalinger for mindre stillesittende tid. Oppgaven bruker den veletablerte designvitenskapelige forskningsmetodikken for å utvikle teoretiske grunnlag og praktiske implementeringer. Samlet sett fokuserer denne forskningen på teknologisk verifisering snarere enn klinisk evaluering.publishedVersio