Using mixed-reality to develop smart environments

Smart homes, smart cars, smart classrooms are now a reality as the world becomes increasingly interconnected by ubiquitous computing technology. The next step is to interconnect such environments, however there are a number of significant barriers to advancing research in this area, most notably the lack of available environments, standards and tools etc. A possible solution is the use of simulated spaces, nevertheless as realistic as strive to make them, they are, at best, only approximations to the real spaces, with important differences such as utilising idealised rather than noisy sensor data. In this respect, an improvement to simulation is emulation, which uses specially adapted physical components to imitate real systems and environments. In this paper we present our work-in-progress towards the creation of a development tool for intelligent environments based on the interconnection of simulated, emulated and real intelligent spaces using a distributed model of mixed reality. To do so, we propose the use of physical/virtual components (xReality objects) able to be combined through a 3D graphical user interface, sharing real-time information. We present three scenarios of interconnected real and emulated spaces, used for education, achieving integration between real and virtual worlds

Context-aware Dynamic Discovery and Configuration of 'Things' in Smart Environments

The Internet of Things (IoT) is a dynamic global information network consisting of Internet-connected objects, such as RFIDs, sensors, actuators, as well as other instruments and smart appliances that are becoming an integral component of the future Internet. Currently, such Internet-connected objects or `things' outnumber both people and computers connected to the Internet and their population is expected to grow to 50 billion in the next 5 to 10 years. To be able to develop IoT applications, such `things' must become dynamically integrated into emerging information networks supported by architecturally scalable and economically feasible Internet service delivery models, such as cloud computing. Achieving such integration through discovery and configuration of `things' is a challenging task. Towards this end, we propose a Context-Aware Dynamic Discovery of {Things} (CADDOT) model. We have developed a tool SmartLink, that is capable of discovering sensors deployed in a particular location despite their heterogeneity. SmartLink helps to establish the direct communication between sensor hardware and cloud-based IoT middleware platforms. We address the challenge of heterogeneity using a plug in architecture. Our prototype tool is developed on an Android platform. Further, we employ the Global Sensor Network (GSN) as the IoT middleware for the proof of concept validation. The significance of the proposed solution is validated using a test-bed that comprises 52 Arduino-based Libelium sensors.Comment: Big Data and Internet of Things: A Roadmap for Smart Environments, Studies in Computational Intelligence book series, Springer Berlin Heidelberg, 201

Automating unobtrusive personalized services in ambient media environments

The final publication is available at Springer via http://dx.doi.org/10.1007/s11042-013-1634-2In the age of ambient media, people are surrounded by lots of physical objects (media objects) for rendering the digital world in the natural environment. These media objects should interact with users in a way that is not disturbing for them. To address this issue, this work presents a design and automation strategy for augmenting the world around us with personalized ambient media services that behave in a considerate manner. That is, ambient services are capable of adjusting its obtrusiveness level (i.e., the extent to which each service intrudes the user¿s mind) by using the appropriate media objects for each user¿s situation.This work has been developed with the support of MICINN, under the project EVERYWARE TIN2010-18011, and the support of the Christian Doppler Forschungsgesellschaft and the BMWFJ, Austria.Serral Asensio, E.; Gil Pascual, M.; Valderas Aranda, PJ.; Pelechano Ferragud, V. (2014). Automating unobtrusive personalized services in ambient media environments. Multimedia Tools and Applications. 71(1):159-178. https://doi.org/10.1007/s11042-013-1634-2S159178711Bencomo N, Grace P, Flores-Cortés CA, Hughes D, Blair GS (2008) Genie: supporting the model driven development of reflective, component-based adaptive systems. In: ICSE, pp 811–814Blumendorf M, Lehmann G, Albayrak S (2010) Bridging models and systems at runtime to build adaptive user interfaces. In: Proc. of EICS 2010. ACM, pp 9–18Brown DM (2010) Communicating design: developing web site documentation for design and planning, 2nd edn. New Riders PressCalinescu R (2011) When the requirements for adaptation and high integrity meet. In: Proceedings of the 8th workshop on assurances for self-adaptive systems, ASAS ’11. ACM, New York, pp 1–4Filieri A, Ghezzi C, Tamburrelli G (2011) Run-time efficient probabilistic model checking. In: Proceedings of the 33rd International Conference on Software Engineering, ICSE ’11. ACM, New York, pp 341–350Gershenfeld N, Krikorian R, Cohen D (2004) The internet of things. Sci Am 291(4):46–51Gibbs WW (2005) Considerate computing. Sci Am 292(1):54–61Gulliksen J, Goransson B, Boivie I, Blomkvist S, Persson J, Cajander A (2003) Key principles for user-centred systems design. Behav Inform Technol 22:397–409Hinckley K, Horvitz E (2001) Toward more sensitive mobile phones. In: Proc. of the UIST ’01, pp 191–192Ho J, Intille SS (2005) Using context-aware computing to reduce the perceived burden of interruptions from mobile devices. In: Proc. of CHI ’05. ACM, pp 909–918Horvitz E, Kadie C, Paek T, Hovel D (2003) Models of attention in computing and communication: from principles to applications. Commun ACM 46:52–59Ju W, Leifer L (2008) The design of implicit interactions: making interactive systems less obnoxious. Des Issues 24(3):72–84Kortuem G, Kawsar F, Fitton D, Sundramoorthy V (2010) Smart objects as building blocks for the internet of things. IEEE Internet Comput 14(1):44–51Lewis JR (1995) Ibm computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum Comput Interact 7(1):57–78Lugmayr A, Risse T, Stockleben B, Laurila K, Kaario J (2009) Semantic ambient media—an introduction. Multimed Tools Appl 43(3):337–359Mattern F (2003) From smart devices to smart everyday objects. In: Proc. Smart Objects Conf. (SOC 03). Springer, pp 15–16Morin B, Barais O, Jezequel JM, Fleurey F, Solberg A (2009) Models run.time to support dynamic adaptation. Comput 42(10):44–51Nelson L, Churchill EF (2005) User experience of physical-digital object systems: implications for representation and infrastructure. Paper presented at smart object systems workshop, in cojunction with ubicomp 2005Paternò F (2002) Concurtasktrees: an engineered approach to model-based design of interactive systems. In: L.E. Associates (ed) The handbook of analysis for human-computer interaction, pp 483–500Paternò F (2003) From model-based to natural development. HCI International, pp 592–596Ramchurn SD, Deitch B, Thompson MK, Roure DCD, Jennings NR, Luck M (2004) Minimising intrusiveness in pervasive computing environments using multi-agent negotiation. MobiQuitous ’04, pp 364–372Runeson P, Höst M (2009) Guidelines for conducting and reporting case study research in software engineering. Empir Softw Eng 14(2):131–164Schmidt A (2000) Implicit human computer interaction through context. Pers Technol 4(2–3):191–199Serral E, Valderas P, Pelechano V (2010) Supporting runtime system evolution to adapt to user behaviour. In: Proc. of CAiSE’10, pp 378–392Serral E, Valderas P, Pelechano V (2010) Towards the model driven development of context-aware pervasive systems. PMC 6(2):254–280Siegemund F (2004) A context-aware communication platform for smart objects. In: Proc of the int conf on pervasive computing. Springer, pp 69–86Streitz NA, Rocker C, Prante T, Alphen Dv, Stenzel R, Magerkurth C (2005) Designing smart artifacts for smart environments. Comput 38(3):41–49. doi: 10.1109/MC.2005.92Thiesse F, Kohler M (2008) An analysis of usage-based pricing policies for smart products. Electron Mark 18(3):232–241. doi: 10.1080/10196780802265751Vastenburg MH, Keyson DV, de Ridder H (2008) Considerate home notification systems: a field study of acceptability of notifications in the home. Pers Ubiquit Comput 12(8):555–56

Mixed reality participants in smart meeting rooms and smart home enviroments

Human–computer interaction requires modeling of the user. A user profile typically contains preferences, interests, characteristics, and interaction behavior. However, in its multimodal interaction with a smart environment the user displays characteristics that show how the user, not necessarily consciously, verbally and nonverbally provides the smart environment with useful input and feedback. Especially in ambient intelligence environments we encounter situations where the environment supports interaction between the environment, smart objects (e.g., mobile robots, smart furniture) and human participants in the environment. Therefore it is useful for the profile to contain a physical representation of the user obtained by multi-modal capturing techniques. We discuss the modeling and simulation of interacting participants in a virtual meeting room, we discuss how remote meeting participants can take part in meeting activities and they have some observations on translating research results to smart home environments

Towards Simulating Humans in Augmented Multi-party Interaction

Human-computer interaction requires modeling of the user. A user profile typically contains preferences, interests, characteristics, and interaction behavior. However, in its multimodal interaction with a smart environment the user displays characteristics that show how the user, not necessarily consciously, verbally and nonverbally provides the smart environment with useful input and feedback. Especially in ambient intelligence environments we encounter situations where the environment supports interaction between the environment, smart objects (e.g., mobile robots, smart furniture) and human participants in the environment. Therefore it is useful for the profile to contain a physical representation of the user obtained by multi-modal capturing techniques. We discuss the modeling and simulation of interacting participants in the European AMI research project

Application and validation of capacitive proximity sensing systems in smart environments

Smart environments feature a number of computing and sensing devices that support occupants in performing their tasks. In the last decades there has been a multitude of advances in miniaturizing sensors and computers, while greatly increasing their performance. As a result new devices are introduced into our daily lives that have a plethora of functions. Gathering information about the occupants is fundamental in adapting the smart environment according to preference and situation. There is a large number of different sensing devices available that can provide information about the user. They include cameras, accelerometers, GPS, acoustic systems, or capacitive sensors. The latter use the properties of an electric field to sense presence and properties of conductive objects within range. They are commonly employed in finger-controlled touch screens that are present in billions of devices. A less common variety is the capacitive proximity sensor. It can detect the presence of the human body over a distance, providing interesting applications in smart environments. Choosing the right sensor technology is an important decision in designing a smart environment application. Apart from looking at previous use cases, this process can be supported by providing more formal methods. In this work I present a benchmarking model that is designed to support this decision process for applications in smart environments. Previous benchmarks for pervasive systems have been adapted towards sensors systems and include metrics that are specific for smart environments. Based on distinct sensor characteristics, different ratings are used as weighting factors in calculating a benchmarking score. The method is verified using popularity matching in two scientific databases. Additionally, there are extensions to cope with central tendency bias and normalization with regards to average feature rating. Four relevant application areas are identified by applying this benchmark to applications in smart environments and capacitive proximity sensors. They are indoor localization, smart appliances, physiological sensing and gesture interaction. Any application area has a set of challenges regarding the required sensor technology, layout of the systems, and processing that can be tackled using various new or improved methods. I will present a collection of existing and novel methods that support processing data generated by capacitive proximity sensors. These are in the areas of sparsely distributed sensors, model-driven fitting methods, heterogeneous sensor systems, image-based processing and physiological signal processing. To evaluate the feasibility of these methods, several prototypes have been created and tested for performance and usability. Six of them are presented in detail. Based on these evaluations and the knowledge generated in the design process, I am able to classify capacitive proximity sensing in smart environments. This classification consists of a comparison to other popular sensing technologies in smart environments, the major benefits of capacitive proximity sensors, and their limitations. In order to support parties interested in developing smart environment applications using capacitive proximity sensors, I present a set of guidelines that support the decision process from technology selection to choice of processing methods

Position paper on realizing smart products: challenges for Semantic Web technologies

In the rapidly developing space of novel technologies that combine sensing and semantic technologies, research on smart products has the potential of establishing a research field in itself. In this paper, we synthesize existing work in this area in order to define and characterize smart products. We then reflect on a set of challenges that semantic technologies are likely to face in this domain. Finally, in order to initiate discussion in the workshop, we sketch an initial comparison of smart products and semantic sensor networks from the perspective of knowledge technologies