Eyewear Computing \u2013 Augmenting the Human with Head-Mounted Wearable Assistants

The seminar was composed of workshops and tutorials on head-mounted eye tracking, egocentric vision, optics, and head-mounted displays. The seminar welcomed 30 academic and industry researchers from Europe, the US, and Asia with a diverse background, including wearable and ubiquitous computing, computer vision, developmental psychology, optics, and human-computer interaction. In contrast to several previous Dagstuhl seminars, we used an ignite talk format to reduce the time of talks to one half-day and to leave the rest of the week for hands-on sessions, group work, general discussions, and socialising. The key results of this seminar are 1) the identification of key research challenges and summaries of breakout groups on multimodal eyewear computing, egocentric vision, security and privacy issues, skill augmentation and task guidance, eyewear computing for gaming, as well as prototyping of VR applications, 2) a list of datasets and research tools for eyewear computing, 3) three small-scale datasets recorded during the seminar, 4) an article in ACM Interactions entitled \u201cEyewear Computers for Human-Computer Interaction\u201d, as well as 5) two follow-up workshops on \u201cEgocentric Perception, Interaction, and Computing\u201d at the European Conference on Computer Vision (ECCV) as well as \u201cEyewear Computing\u201d at the ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp)

An affective computing and image retrieval approach to support diversified and emotion-aware reminiscence therapy sessions

A demência é uma das principais causas de dependência e incapacidade entre as pessoas idosas em todo o mundo. A terapia de reminiscência é uma terapia não farmacológica comummente utilizada nos cuidados com demência devido ao seu valor terapêutico para as pessoas com demência. Esta terapia é útil para criar uma comunicação envolvente entre pessoas com demência e o resto do mundo, utilizando as capacidades preservadas da memória a longo prazo, em vez de enfatizar as limitações existentes por forma a aliviar a experiência de fracasso e isolamento social. As soluções tecnológicas de assistência existentes melhoram a terapia de reminiscência ao proporcionar uma experiência mais envolvente para todos os participantes (pessoas com demência, familiares e clínicos), mas não estão livres de lacunas: a) os dados multimédia utilizados permanecem inalterados ao longo das sessões, e há uma falta de personalização para cada pessoa com demência; b) não têm em conta as emoções transmitidas pelos dados multimédia utilizados nem as reacções emocionais da pessoa com demência aos dados multimédia apresentados; c) a perspectiva dos cuidadores ainda não foi totalmente tida em consideração. Para superar estes desafios, seguimos uma abordagem de concepção centrada no utilizador através de inquéritos mundiais, entrevistas de seguimento, e grupos de discussão com cuidadores formais e informais para informar a concepção de soluções tecnológicas no âmbito dos cuidados de demência. Para cumprir com os requisitos identificados, propomos novos métodos que facilitam a inclusão de emoções no loop durante a terapia de reminiscência para personalizar e diversificar o conteúdo das sessões ao longo do tempo. As contribuições desta tese incluem: a) um conjunto de requisitos funcionais validados recolhidos com os cuidadores formais e informais, os resultados esperados com o cumprimento de cada requisito, e um modelo de arquitectura para o desenvolvimento de soluções tecnológicas de assistência para cuidados de demência; b) uma abordagem end-to-end para identificar automaticamente múltiplas informações emocionais transmitidas por imagens; c) uma abordagem para reduzir a quantidade de imagens que precisam ser anotadas pelas pessoas sem comprometer o desempenho dos modelos de reconhecimento; d) uma técnica de fusão tardia interpretável que combina dinamicamente múltiplos sistemas de recuperação de imagens com base em conteúdo para procurar eficazmente por imagens semelhantes para diversificar e personalizar o conjunto de imagens disponíveis para serem utilizadas nas sessões.Dementia is one of the major causes of dependency and disability among elderly subjects worldwide. Reminiscence therapy is an inexpensive non-pharmacological therapy commonly used within dementia care due to its therapeutic value for people with dementia. This therapy is useful to create engaging communication between people with dementia and the rest of the world by using the preserved abilities of long-term memory rather than emphasizing the existing impairments to alleviate the experience of failure and social isolation. Current assistive technological solutions improve reminiscence therapy by providing a more lively and engaging experience to all participants (people with dementia, family members, and clinicians), but they are not free of drawbacks: a) the multimedia data used remains unchanged throughout sessions, and there is a lack of customization for each person with dementia; b) they do not take into account the emotions conveyed by the multimedia data used nor the person with dementia’s emotional reactions to the multimedia presented; c) the caregivers’ perspective have not been fully taken into account yet. To overcome these challenges, we followed a usercentered design approach through worldwide surveys, follow-up interviews, and focus groups with formal and informal caregivers to inform the design of technological solutions within dementia care. To fulfil the requirements identified, we propose novel methods that facilitate the inclusion of emotions in the loop during reminiscence therapy to personalize and diversify the content of the sessions over time. Contributions from this thesis include: a) a set of validated functional requirements gathered from formal and informal caregivers, the expected outcomes with the fulfillment of each requirement, and an architecture’s template for the development of assistive technology solutions for dementia care; b) an end-to-end approach to automatically identify multiple emotional information conveyed by images; c) an approach to reduce the amount of images that need to be annotated by humans without compromising the recognition models’ performance; d) an interpretable late-fusion technique that dynamically combines multiple content-based image retrieval systems to effectively search for similar images to diversify and personalize the pool of images available to be used in sessions

Beyond mobile apps: a survey of technologies for mental well-being

Mental health problems are on the rise globally and strain national health systems worldwide. Mental disorders are closely associated with fear of stigma, structural barriers such as financial burden, and lack of available services and resources which often prohibit the delivery of frequent clinical advice and monitoring. Technologies for mental well-being exhibit a range of attractive properties, which facilitate the delivery of state-of-the-art clinical monitoring. This review article provides an overview of traditional techniques followed by their technological alternatives, sensing devices, behaviour changing tools, and feedback interfaces. The challenges presented by these technologies are then discussed with data collection, privacy, and battery life being some of the key issues which need to be carefully considered for the successful deployment of mental health toolkits. Finally, the opportunities this growing research area presents are discussed including the use of portable tangible interfaces combining sensing and feedback technologies. Capitalising on the data these ubiquitous devices can record, state of the art machine learning algorithms can lead to the development of robust clinical decision support tools towards diagnosis and improvement of mental well-being delivery in real-time

An overview of video recommender systems: state-of-the-art and research issues

Video platforms have become indispensable components within a diverse range of applications, serving various purposes in entertainment, e-learning, corporate training, online documentation, and news provision. As the volume and complexity of video content continue to grow, the need for personalized access features becomes an inevitable requirement to ensure efficient content consumption. To address this need, recommender systems have emerged as helpful tools providing personalized video access. By leveraging past user-specific video consumption data and the preferences of similar users, these systems excel in recommending videos that are highly relevant to individual users. This article presents a comprehensive overview of the current state of video recommender systems (VRS), exploring the algorithms used, their applications, and related aspects. In addition to an in-depth analysis of existing approaches, this review also addresses unresolved research challenges within this domain. These unexplored areas offer exciting opportunities for advancements and innovations, aiming to enhance the accuracy and effectiveness of personalized video recommendations. Overall, this article serves as a valuable resource for researchers, practitioners, and stakeholders in the video domain. It offers insights into cutting-edge algorithms, successful applications, and areas that merit further exploration to advance the field of video recommendation

A Review on MAS-Based Sentiment and Stress Analysis User-Guiding and Risk-Prevention Systems in Social Network Analysis

[EN] In the current world we live immersed in online applications, being one of the most present of them Social Network Sites (SNSs), and different issues arise from this interaction. Therefore, there is a need for research that addresses the potential issues born from the increasing user interaction when navigating. For this reason, in this survey we explore works in the line of prevention of risks that can arise from social interaction in online environments, focusing on works using Multi-Agent System (MAS) technologies. For being able to assess what techniques are available for prevention, works in the detection of sentiment polarity and stress levels of users in SNSs will be reviewed. We review with special attention works using MAS technologies for user recommendation and guiding. Through the analysis of previous approaches on detection of the user state and risk prevention in SNSs we elaborate potential future lines of work that might lead to future applications where users can navigate and interact between each other in a more safe way.This work was funded by the project TIN2017-89156-R of the Spanish government.Aguado-Sarrió, G.; Julian Inglada, VJ.; García-Fornes, A.; Espinosa Minguet, AR. (2020). A Review on MAS-Based Sentiment and Stress Analysis User-Guiding and Risk-Prevention Systems in Social Network Analysis. Applied Sciences. 10(19):1-29. https://doi.org/10.3390/app10196746S1291019Vanderhoven, E., Schellens, T., Vanderlinde, R., & Valcke, M. (2015). Developing educational materials about risks on social network sites: a design based research approach. Educational Technology Research and Development, 64(3), 459-480. doi:10.1007/s11423-015-9415-4Teens and ICT: Risks and Opportunities. Belgium: TIRO http://www.belspo.be/belspo/fedra/proj.asp?l=en&COD=TA/00/08Risks and Safety on the Internet: The Perspective of European Children: Full Findings and Policy Implications From the EU Kids Online Survey of 9–16 Year Olds and Their Parents in 25 Countries http://eprints.lse.ac.uk/33731/Vanderhoven, E., Schellens, T., & Valcke, M. (2014). Educating teens about the risks on social network sites. An intervention study in Secondary Education. Comunicar, 22(43), 123-132. doi:10.3916/c43-2014-12Christofides, E., Muise, A., & Desmarais, S. (2012). Risky Disclosures on Facebook. Journal of Adolescent Research, 27(6), 714-731. doi:10.1177/0743558411432635George, J. M., & Dane, E. (2016). Affect, emotion, and decision making. Organizational Behavior and Human Decision Processes, 136, 47-55. doi:10.1016/j.obhdp.2016.06.004Thelwall, M. (2017). TensiStrength: Stress and relaxation magnitude detection for social media texts. Information Processing & Management, 53(1), 106-121. doi:10.1016/j.ipm.2016.06.009Thelwall, M., Buckley, K., Paltoglou, G., Cai, D., & Kappas, A. (2010). Sentiment strength detection in short informal text. Journal of the American Society for Information Science and Technology, 61(12), 2544-2558. doi:10.1002/asi.21416Shoumy, N. J., Ang, L.-M., Seng, K. P., Rahaman, D. M. M., & Zia, T. (2020). Multimodal big data affective analytics: A comprehensive survey using text, audio, visual and physiological signals. Journal of Network and Computer Applications, 149, 102447. doi:10.1016/j.jnca.2019.102447Zhang, C., Zeng, D., Li, J., Wang, F.-Y., & Zuo, W. (2009). Sentiment analysis of Chinese documents: From sentence to document level. Journal of the American Society for Information Science and Technology, 60(12), 2474-2487. doi:10.1002/asi.21206Lu, B., Ott, M., Cardie, C., & Tsou, B. K. (2011). Multi-aspect Sentiment Analysis with Topic Models. 2011 IEEE 11th International Conference on Data Mining Workshops. doi:10.1109/icdmw.2011.125Nasukawa, T., & Yi, J. (2003). Sentiment analysis. Proceedings of the international conference on Knowledge capture - K-CAP ’03. doi:10.1145/945645.945658Borth, D., Ji, R., Chen, T., Breuel, T., & Chang, S.-F. (2013). Large-scale visual sentiment ontology and detectors using adjective noun pairs. Proceedings of the 21st ACM international conference on Multimedia - MM ’13. doi:10.1145/2502081.2502282Deb, S., & Dandapat, S. (2019). Emotion Classification Using Segmentation of Vowel-Like and Non-Vowel-Like Regions. IEEE Transactions on Affective Computing, 10(3), 360-373. doi:10.1109/taffc.2017.2730187Deng, J., Zhang, Z., Marchi, E., & Schuller, B. (2013). Sparse Autoencoder-Based Feature Transfer Learning for Speech Emotion Recognition. 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction. doi:10.1109/acii.2013.90Nicolaou, M. A., Gunes, H., & Pantic, M. (2011). Continuous Prediction of Spontaneous Affect from Multiple Cues and Modalities in Valence-Arousal Space. IEEE Transactions on Affective Computing, 2(2), 92-105. doi:10.1109/t-affc.2011.9Hossain, M. S., Muhammad, G., Alhamid, M. F., Song, B., & Al-Mutib, K. (2016). Audio-Visual Emotion Recognition Using Big Data Towards 5G. Mobile Networks and Applications, 21(5), 753-763. doi:10.1007/s11036-016-0685-9Zhou, F., Jianxin Jiao, R., & Linsey, J. S. (2015). Latent Customer Needs Elicitation by Use Case Analogical Reasoning From Sentiment Analysis of Online Product Reviews. Journal of Mechanical Design, 137(7). doi:10.1115/1.4030159Ceci, F., Goncalves, A. L., & Weber, R. (2016). A model for sentiment analysis based on ontology and cases. IEEE Latin America Transactions, 14(11), 4560-4566. doi:10.1109/tla.2016.7795829Vizer, L. M., Zhou, L., & Sears, A. (2009). Automated stress detection using keystroke and linguistic features: An exploratory study. International Journal of Human-Computer Studies, 67(10), 870-886. doi:10.1016/j.ijhcs.2009.07.005Feldman, R. (2013). Techniques and applications for sentiment analysis. Communications of the ACM, 56(4), 82-89. doi:10.1145/2436256.2436274Schouten, K., & Frasincar, F. (2016). Survey on Aspect-Level Sentiment Analysis. IEEE Transactions on Knowledge and Data Engineering, 28(3), 813-830. doi:10.1109/tkde.2015.2485209Ji, R., Cao, D., Zhou, Y., & Chen, F. (2016). Survey of visual sentiment prediction for social media analysis. Frontiers of Computer Science, 10(4), 602-611. doi:10.1007/s11704-016-5453-2Li, L., Cao, D., Li, S., & Ji, R. (2015). Sentiment analysis of Chinese micro-blog based on multi-modal correlation model. 2015 IEEE International Conference on Image Processing (ICIP). doi:10.1109/icip.2015.7351718Lee, P.-M., Tsui, W.-H., & Hsiao, T.-C. (2015). The Influence of Emotion on Keyboard Typing: An Experimental Study Using Auditory Stimuli. PLOS ONE, 10(6), e0129056. doi:10.1371/journal.pone.0129056Matsiola, M., Dimoulas, C., Kalliris, G., & Veglis, A. A. (2018). Augmenting User Interaction Experience Through Embedded Multimodal Media Agents in Social Networks. Information Retrieval and Management, 1972-1993. doi:10.4018/978-1-5225-5191-1.ch088Rosaci, D. (2007). CILIOS: Connectionist inductive learning and inter-ontology similarities for recommending information agents. Information Systems, 32(6), 793-825. doi:10.1016/j.is.2006.06.003Buccafurri, F., Comi, A., Lax, G., & Rosaci, D. (2016). Experimenting with Certified Reputation in a Competitive Multi-Agent Scenario. IEEE Intelligent Systems, 31(1), 48-55. doi:10.1109/mis.2015.98Rosaci, D., & Sarnè, G. M. L. (2014). Multi-agent technology and ontologies to support personalization in B2C E-Commerce. Electronic Commerce Research and Applications, 13(1), 13-23. doi:10.1016/j.elerap.2013.07.003Singh, A., & Sharma, A. (2017). MAICBR: A Multi-agent Intelligent Content-Based Recommendation System. Lecture Notes in Networks and Systems, 399-411. doi:10.1007/978-981-10-3920-1_41Villavicencio, C., Schiaffino, S., Diaz-Pace, J. A., Monteserin, A., Demazeau, Y., & Adam, C. (2016). A MAS Approach for Group Recommendation Based on Negotiation Techniques. Lecture Notes in Computer Science, 219-231. doi:10.1007/978-3-319-39324-7_19Rincon, J. A., de la Prieta, F., Zanardini, D., Julian, V., & Carrascosa, C. (2017). Influencing over people with a social emotional model. Neurocomputing, 231, 47-54. doi:10.1016/j.neucom.2016.03.107Aguado, G., Julian, V., Garcia-Fornes, A., & Espinosa, A. (2020). A Multi-Agent System for guiding users in on-line social environments. Engineering Applications of Artificial Intelligence, 94, 103740. doi:10.1016/j.engappai.2020.103740Aguado, G., Julián, V., García-Fornes, A., & Espinosa, A. (2020). Using Keystroke Dynamics in a Multi-Agent System for User Guiding in Online Social Networks. Applied Sciences, 10(11), 3754. doi:10.3390/app10113754Camara, M., Bonham-Carter, O., & Jumadinova, J. (2015). A multi-agent system with reinforcement learning agents for biomedical text mining. Proceedings of the 6th ACM Conference on Bioinformatics, Computational Biology and Health Informatics. doi:10.1145/2808719.2812596Lombardo, G., Fornacciari, P., Mordonini, M., Tomaiuolo, M., & Poggi, A. (2019). A Multi-Agent Architecture for Data Analysis. Future Internet, 11(2), 49. doi:10.3390/fi11020049Schweitzer, F., & Garcia, D. (2010). An agent-based model of collective emotions in online communities. The European Physical Journal B, 77(4), 533-545. doi:10.1140/epjb/e2010-00292-