Self-adaptive unobtrusive interactions of mobile computing systems

[EN] In Pervasive Computing environments, people are surrounded by a lot of embedded services. Since pervasive devices, such as mobile devices, have become a key part of our everyday life, they enable users to always be connected to the environment, making demands on one of the most valuable resources of users: human attention. A challenge of the mobile computing systems is regulating the request for users¿ attention. In other words, service interactions should behave in a considerate manner by taking into account the degree to which each service intrudes on the user¿s mind (i.e., the degree of obtrusiveness). The main goal of this paper is to introduce self-adaptive capabilities in mobile computing systems in order to provide non-disturbing interactions. We achieve this by means of an software infrastructure that automatically adapts the service interaction obtrusiveness according to the user¿s context. This infrastructure works from a set of high-level models that define the unobtrusive adaptation behavior and its implication with the interaction resources in a technology-independent way. Our infrastructure has been validated through several experiments to assess its correctness, performance, and the achieved user experience through a user study.This work has been developed with the support of MINECO under the project SMART-ADAPT TIN2013-42981-P, and co-financed by the Generalitat Valenciana under the postdoctoral fellowship APOSTD/2016/042.Gil Pascual, M.; Pelechano Ferragud, V. (2017). Self-adaptive unobtrusive interactions of mobile computing systems. Journal of Ambient Intelligence and Smart Environments. 9(6):659-688. https://doi.org/10.3233/AIS-170463S65968896Aleksy, M., Butter, T., & Schader, M. (2008). Context-Aware Loading for Mobile Applications. Lecture Notes in Computer Science, 12-20. doi:10.1007/978-3-540-85693-1_3Y. Bachvarova, B. van Dijk and A. Nijholt, Towards a unified knowledge-based approach to modality choice, in: Proc. Workshop on Multimodal Output Generation (MOG), 2007, pp. 5–15.Barkhuus, L., & Dey, A. (2003). Is Context-Aware Computing Taking Control away from the User? Three Levels of Interactivity Examined. Lecture Notes in Computer Science, 149-156. doi:10.1007/978-3-540-39653-6_12Bellotti, V., & Edwards, K. (2001). Intelligibility and Accountability: Human Considerations in Context-Aware Systems. Human–Computer Interaction, 16(2-4), 193-212. doi:10.1207/s15327051hci16234_05D. Benavides, P. Trinidad and A. Ruiz-Cortés, Automated reasoning on feature models, in: Proceedings of the 17th International Conference on Advanced Information Systems Engineering, CAiSE’05, Springer-Verlag, Berlin, 2005, pp. 491–503.Bernsen, N. O. (1994). Foundations of multimodal representations: a taxonomy of representational modalities. Interacting with Computers, 6(4), 347-371. doi:10.1016/0953-5438(94)90008-6Bettini, C., Brdiczka, O., Henricksen, K., Indulska, J., Nicklas, D., Ranganathan, A., & Riboni, D. (2010). A survey of context modelling and reasoning techniques. Pervasive and Mobile Computing, 6(2), 161-180. doi:10.1016/j.pmcj.2009.06.002Blumendorf, M., Lehmann, G., & Albayrak, S. (2010). Bridging models and systems at runtime to build adaptive user interfaces. Proceedings of the 2nd ACM SIGCHI symposium on Engineering interactive computing systems - EICS ’10. doi:10.1145/1822018.1822022D.M. Brown, Communicating Design: Developing Web Site Documentation for Design and Planning, 2nd edn, New Riders Press, 2010.J. Bruin, Statistical Analyses Using SPSS, 2011, http://www.ats.ucla.edu/stat/spss/whatstat/whatstat.htm#1sampt.J. Cámara, G. Moreno and D. Garlan, Reasoning about human participation in self-adaptive systems, in: SEAMS 2015, 2015, pp. 146–156.Campbell, A., & Choudhury, T. (2012). From Smart to Cognitive Phones. IEEE Pervasive Computing, 11(3), 7-11. doi:10.1109/mprv.2012.41Y. Cao, M. Theune and A. Nijholt, Modality effects on cognitive load and performance in high-load information presentation, in: Proceedings of the 14th International Conference on Intelligent User Interfaces, IUI’09, ACM, New York, 2009, pp. 335–344.Chang, F., & Ren, J. (2007). Validating system properties exhibited in execution traces. Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering - ASE ’07. doi:10.1145/1321631.1321723H. Chen and J.P. Black, A quantitative approach to non-intrusive computing, in: Mobiquitous’08: Proceedings of the 5th Annual International Conference on Mobile and Ubiquitous Systems, ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), ICST, Brussels, 2008, pp. 1–10.Chittaro, L. (2010). Distinctive aspects of mobile interaction and their implications for the design of multimodal interfaces. Journal on Multimodal User Interfaces, 3(3), 157-165. doi:10.1007/s12193-010-0036-2Clerckx, T., Vandervelpen, C., & Coninx, K. (2008). Task-Based Design and Runtime Support for Multimodal User Interface Distribution. Lecture Notes in Computer Science, 89-105. doi:10.1007/978-3-540-92698-6_6Cook, D. J., & Das, S. K. (2012). Pervasive computing at scale: Transforming the state of the art. Pervasive and Mobile Computing, 8(1), 22-35. doi:10.1016/j.pmcj.2011.10.004Cornelissen, B., Zaidman, A., van Deursen, A., Moonen, L., & Koschke, R. (2009). A Systematic Survey of Program Comprehension through Dynamic Analysis. IEEE Transactions on Software Engineering, 35(5), 684-702. doi:10.1109/tse.2009.28Czarnecki, K. (2004). Generative Software Development. Lecture Notes in Computer Science, 321-321. doi:10.1007/978-3-540-28630-1_33M. de Sá, C. Duarte, L. Carriço and T. Reis, Designing mobile multimodal applications, in: Information Science Reference, 2010, pp. 106–136, Chapter 5.C. Duarte and L. Carriço, A conceptual framework for developing adaptive multimodal applications, in: Proceedings of the 11th International Conference on Intelligent User Interfaces, IUI’06, ACM, New York, 2006, pp. 132–139.Evers, C., Kniewel, R., Geihs, K., & Schmidt, L. (2014). The user in the loop: Enabling user participation for self-adaptive applications. Future Generation Computer Systems, 34, 110-123. doi:10.1016/j.future.2013.12.010Fagin, R., Halpern, J. Y., & Megiddo, N. (1990). A logic for reasoning about probabilities. Information and Computation, 87(1-2), 78-128. doi:10.1016/0890-5401(90)90060-uFerscha, A. (2012). 20 Years Past Weiser: What’s Next? IEEE Pervasive Computing, 11(1), 52-61. doi:10.1109/mprv.2011.78Floch, J., Frà, C., Fricke, R., Geihs, K., Wagner, M., Lorenzo, J., … Scholz, U. (2012). Playing MUSIC - building context-aware and self-adaptive mobile applications. Software: Practice and Experience, 43(3), 359-388. doi:10.1002/spe.2116Gibbs, W. W. (2005). Considerate Computing. Scientific American, 292(1), 54-61. doi:10.1038/scientificamerican0105-54Gil, M., Giner, P., & Pelechano, V. (2011). Personalization for unobtrusive service interaction. Personal and Ubiquitous Computing, 16(5), 543-561. doi:10.1007/s00779-011-0414-0Gil Pascual, M. (s. f.). Adapting Interaction Obtrusiveness: Making Ubiquitous Interactions Less Obnoxious. A Model Driven Engineering approach. doi:10.4995/thesis/10251/31660Haapalainen, E., Kim, S., Forlizzi, J. F., & Dey, A. K. (2010). Psycho-physiological measures for assessing cognitive load. Proceedings of the 12th ACM international conference on Ubiquitous computing - Ubicomp ’10. doi:10.1145/1864349.1864395Hallsteinsen, S., Geihs, K., Paspallis, N., Eliassen, F., Horn, G., Lorenzo, J., … Papadopoulos, G. A. (2012). A development framework and methodology for self-adapting applications in ubiquitous computing environments. Journal of Systems and Software, 85(12), 2840-2859. doi:10.1016/j.jss.2012.07.052Hassenzahl, M. (2004). The Interplay of Beauty, Goodness, and Usability in Interactive Products. Human-Computer Interaction, 19(4), 319-349. doi:10.1207/s15327051hci1904_2Hassenzahl, M., & Tractinsky, N. (2006). User experience - a research agenda. Behaviour & Information Technology, 25(2), 91-97. doi:10.1080/01449290500330331Ho, J., & Intille, S. S. (2005). Using context-aware computing to reduce the perceived burden of interruptions from mobile devices. Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ’05. doi:10.1145/1054972.1055100Horvitz, E., Kadie, C., Paek, T., & Hovel, D. (2003). Models of attention in computing and communication. Communications of the ACM, 46(3), 52. doi:10.1145/636772.636798Horvitz, E., Koch, P., Sarin, R., Apacible, J., & Subramani, M. (2005). Bayesphone: Precomputation of Context-Sensitive Policies for Inquiry and Action in Mobile Devices. Lecture Notes in Computer Science, 251-260. doi:10.1007/11527886_33Kephart, J. O., & Chess, D. M. (2003). The vision of autonomic computing. Computer, 36(1), 41-50. doi:10.1109/mc.2003.1160055Korpipaa, P., Malm, E.-J., Rantakokko, T., Kyllonen, V., Kela, J., Mantyjarvi, J., … Kansala, I. (2006). Customizing User Interaction in Smart Phones. IEEE Pervasive Computing, 5(3), 82-90. doi:10.1109/mprv.2006.49S. Lemmelä, A. Vetek, K. Mäkelä and D. Trendafilov, Designing and evaluating multimodal interaction for mobile contexts, in: Proceedings of the 10th International Conference on Multimodal Interfaces, ICMI’08, ACM, New York, 2008, pp. 265–272.Lim, B. Y. (2010). Improving trust in context-aware applications with intelligibility. Proceedings of the 12th ACM international conference adjunct papers on Ubiquitous computing - Ubicomp ’10. doi:10.1145/1864431.1864491J.-Y. Mao, K. Vredenburg, P.W. Smith and T. Carey, User-centered design methods in practice: A survey of the state of the art, in: Proceedings of the 2001 Conference of the Centre for Advanced Studies on Collaborative Research, CASCON’01, IBM Press, 2001, p. 12.Maoz, S. (2009). Using Model-Based Traces as Runtime Models. Computer, 42(10), 28-36. doi:10.1109/mc.2009.336Mayer, R. E., & Moreno, R. (2003). Nine Ways to Reduce Cognitive Load in Multimedia Learning. Educational Psychologist, 38(1), 43-52. doi:10.1207/s15326985ep3801_6Motti, V. G., & Vanderdonckt, J. (2013). A computational framework for context-aware adaptation of user interfaces. IEEE 7th International Conference on Research Challenges in Information Science (RCIS). doi:10.1109/rcis.2013.6577709R. Murch, Autonomic Computing, IBM Press, 2004.Obrenovic, Z., Abascal, J., & Starcevic, D. (2007). Universal accessibility as a multimodal design issue. Communications of the ACM, 50(5), 83-88. doi:10.1145/1230819.1241668Patterson, D. J., Baker, C., Ding, X., Kaufman, S. J., Liu, K., & Zaldivar, A. (2008). Online everywhere. Proceedings of the 10th international conference on Ubiquitous computing - UbiComp ’08. doi:10.1145/1409635.1409645Pielot, M., de Oliveira, R., Kwak, H., & Oliver, N. (2014). Didn’t you see my message? Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI ’14. doi:10.1145/2556288.2556973Poppinga, B., Heuten, W., & Boll, S. (2014). Sensor-Based Identification of Opportune Moments for Triggering Notifications. IEEE Pervasive Computing, 13(1), 22-29. doi:10.1109/mprv.2014.15S. Ramchurn, B. Deitch, M. Thompson, D. De Roure, N. Jennings and M. Luck, Minimising intrusiveness in pervasive computing environments using multi-agent negotiation, in: Mobile and Ubiquitous Systems: Networking and Services, MOBIQUITOUS 2004. The First Annual International Conference on, 2004, pp. 364–371.C. Roda, Human Attention and Its Implications for Human-Computer Interaction, Cambridge University Press, 2011.S. Rosenthal, A.K. Dey and M. Veloso, Using decision-theoretic experience sampling to build personalized mobile phone interruption models, in: Proceedings of the 9th International Conference on Pervasive Computing, Pervasive 2011, Springer-Verlag, Berlin, 2011, pp. 170–187.E. Rukzio, K. Leichtenstern and V. Callaghan, An experimental comparison of physical mobile interaction techniques: Touching, pointing and scanning, in: 8th International Conference on Ubiquitous Computing, UbiComp 2006, Orange County, California, 2006.Serral, E., Valderas, P., & Pelechano, V. (2010). Towards the Model Driven Development of context-aware pervasive systems. Pervasive and Mobile Computing, 6(2), 254-280. doi:10.1016/j.pmcj.2009.07.006D. Siewiorek, A. Smailagic, J. Furukawa, A. Krause, N. Moraveji, K. Reiger, J. Shaffer and F.L. Wong, Sensay: A context-aware mobile phone, in: Proceedings of the 7th IEEE International Symposium on Wearable Computers, ISWC’03, IEEE Computer Society, Washington, 2003, p. 248.Tedre, M. (2006). What should be automated? Proceedings of the 1st ACM international workshop on Human-centered multimedia - HCM ’06. doi:10.1145/1178745.1178753M. Valtonen, A.-M. Vainio and J. Vanhala, Proactive and adaptive fuzzy profile control for mobile phones, in: IEEE International Conference on Pervasive Computing and Communications, 2009, PerCom, 2009, pp. 1–3.Vastenburg, M. H., Keyson, D. V., & de Ridder, H. (2007). Considerate home notification systems: a field study of acceptability of notifications in the home. Personal and Ubiquitous Computing, 12(8), 555-566. doi:10.1007/s00779-007-0176-xWarnock, D., McGee-Lennon, M., & Brewster, S. (2011). The Role of Modality in Notification Performance. Lecture Notes in Computer Science, 572-588. doi:10.1007/978-3-642-23771-3_43Weiser, M., & Brown, J. S. (1997). The Coming Age of Calm Technology. Beyond Calculation, 75-85. doi:10.1007/978-1-4612-0685-9_6Van Woensel, W., Gil, M., Casteleyn, S., Serral, E., & Pelechano, V. (2013). Adapting the Obtrusiveness of Service Interactions in Dynamically Discovered Environments. Mobile and Ubiquitous Systems: Computing, Networking, and Services, 250-262. doi:10.1007/978-3-642-40238-8_2

Security analysis and exploitation of arduino devices in the internet of things

The pervasive presence of interconnected objects enables new communication paradigms where devices can easily reach each other while interacting within their environment. The so-called Internet of Things (IoT) represents the integration of several computing and communications systems aiming at facilitating the interaction between these devices. Arduino is one of the most popular platforms used to prototype new IoT devices due to its open, flexible and easy-to-use architecture. Ardunio Yun is a dual board microcontroller that supports a Linux distribution and it is currently one of the most versatile and powerful Arduino systems. This feature positions Arduino Yun as a popular platform for developers, but it also introduces unique infection vectors from the security viewpoint. In this work, we present a security analysis of Arduino Yun. We show that Arduino Yun is vulnerable to a number of attacks and we implement a proof of concept capable of exploiting some of them

Anticipatory Mobile Computing: A Survey of the State of the Art and Research Challenges

Today's mobile phones are far from mere communication devices they were ten years ago. Equipped with sophisticated sensors and advanced computing hardware, phones can be used to infer users' location, activity, social setting and more. As devices become increasingly intelligent, their capabilities evolve beyond inferring context to predicting it, and then reasoning and acting upon the predicted context. This article provides an overview of the current state of the art in mobile sensing and context prediction paving the way for full-fledged anticipatory mobile computing. We present a survey of phenomena that mobile phones can infer and predict, and offer a description of machine learning techniques used for such predictions. We then discuss proactive decision making and decision delivery via the user-device feedback loop. Finally, we discuss the challenges and opportunities of anticipatory mobile computing.Comment: 29 pages, 5 figure

Context for Ubiquitous Data Management

In response to the advance of ubiquitous computing technologies, we believe that for computer systems to be ubiquitous, they must be context-aware. In this paper, we address the impact of context-awareness on ubiquitous data management. To do this, we overview different characteristics of context in order to develop a clear understanding of context, as well as its implications and requirements for context-aware data management. References to recent research activities and applicable techniques are also provided

Unobtrusive and pervasive video-based eye-gaze tracking

Eye-gaze tracking has long been considered a desktop technology that finds its use inside the traditional office setting, where the operating conditions may be controlled. Nonetheless, recent advancements in mobile technology and a growing interest in capturing natural human behaviour have motivated an emerging interest in tracking eye movements within unconstrained real-life conditions, referred to as pervasive eye-gaze tracking. This critical review focuses on emerging passive and unobtrusive video-based eye-gaze tracking methods in recent literature, with the aim to identify different research avenues that are being followed in response to the challenges of pervasive eye-gaze tracking. Different eye-gaze tracking approaches are discussed in order to bring out their strengths and weaknesses, and to identify any limitations, within the context of pervasive eye-gaze tracking, that have yet to be considered by the computer vision community.peer-reviewe

Emotions in context: examining pervasive affective sensing systems, applications, and analyses

Pervasive sensing has opened up new opportunities for measuring our feelings and understanding our behavior by monitoring our affective states while mobile. This review paper surveys pervasive affect sensing by examining and considering three major elements of affective pervasive systems, namely; “sensing”, “analysis”, and “application”. Sensing investigates the different sensing modalities that are used in existing real-time affective applications, Analysis explores different approaches to emotion recognition and visualization based on different types of collected data, and Application investigates different leading areas of affective applications. For each of the three aspects, the paper includes an extensive survey of the literature and finally outlines some of challenges and future research opportunities of affective sensing in the context of pervasive computing