Video browsing interfaces and applications: a review

We present a comprehensive review of the state of the art in video browsing and retrieval systems, with special emphasis on interfaces and applications. There has been a significant increase in activity (e.g., storage, retrieval, and sharing) employing video data in the past decade, both for personal and professional use. The ever-growing amount of video content available for human consumption and the inherent characteristics of video data—which, if presented in its raw format, is rather unwieldy and costly—have become driving forces for the development of more effective solutions to present video contents and allow rich user interaction. As a result, there are many contemporary research efforts toward developing better video browsing solutions, which we summarize. We review more than 40 different video browsing and retrieval interfaces and classify them into three groups: applications that use video-player-like interaction, video retrieval applications, and browsing solutions based on video surrogates. For each category, we present a summary of existing work, highlight the technical aspects of each solution, and compare them against each other

Understanding the Role of Interactivity and Explanation in Adaptive Experiences

Adaptive experiences have been an active area of research in the past few decades, accompanied by advances in technology such as machine learning and artificial intelligence. Whether the currently ongoing research on adaptive experiences has focused on personalization algorithms, explainability, user engagement, or privacy and security, there is growing interest and resources in developing and improving these research focuses. Even though the research on adaptive experiences has been dynamic and rapidly evolving, achieving a high level of user engagement in adaptive experiences remains a challenge. %????? This dissertation aims to uncover ways to engage users in adaptive experiences by incorporating interactivity and explanation through four studies. Study I takes the first step to link the explanation and interactivity in machine learning systems to facilitate users\u27 engagement with the underlying machine learning model with the Tic-Tac-Toe game as a use case. The results show that explainable machine learning (XML) systems (and arguably XAI systems in general) indeed benefit from mechanisms that allow users to interact with the system\u27s internal decision rules. Study II, III, and IV further focus on adaptive experiences in recommender systems in specific, exploring the role of interactivity and explanation to keep the user “in-the-loop” in recommender systems, trying to mitigate the ``filter bubble\u27\u27 problem and help users in self-actualizing by supporting them in exploring and understanding their unique tastes. Study II investigates the effect of recommendation source (a human expert vs. an AI algorithm) and justification method (needs-based vs. interest-based justification) on professional development recommendations in a scenario-based study setting. The results show an interaction effect between these two system aspects: users who are told that the recommendations are based on their interests have a better experience when the recommendations are presented as originating from an AI algorithm, while users who are told that the recommendations are based on their needs have a better experience when the recommendations are presented as originating from a human expert. This work implies that while building the proposed novel movie recommender system covered in study IV, it would provide a better user experience if the movie recommendations are presented as originating from algorithms rather than from a human expert considering that movie preferences (which will be visualized by the movies\u27 emotion feature) are usually based on users\u27 interest. Study III explores the effects of four novel alternative recommendation lists on participants’ perceptions of recommendations and their satisfaction with the system. The four novel alternative recommendation lists (RSSA features) which have the potential to go beyond the traditional top N recommendations provide transparency from a different level --- how much else does the system learn about users beyond the traditional top N recommendations, which in turn enable users to interact with these alternative lists by rating the initial recommendations so as to correct or confirm the system\u27s estimates of the alternative recommendations. The subjective evaluation and behavioral analysis demonstrate that the proposed RSSA features had a significant effect on the user experience, surprisingly, two of the four RSSA features (the controversial and hate features) perform worse than the traditional top-N recommendations on the measured subjective dependent variables while the other two RSSA features (the hipster and no clue items) perform equally well and even slightly better than the traditional top-N (but this effect is not statistically significant). Moreover, the results indicate that individual differences, such as the need for novelty and domain knowledge, play a significant role in users’ perception of and interaction with the system. Study IV further combines diversification, visualization, and interactivity, aiming to encourage users to be more engaged with the system. The results show that introducing emotion as an item feature into recommender systems does help in personalization and individual taste exploration; these benefits are greatly optimized through the mechanisms that diversify recommendations by emotional signature, visualize recommendations on the emotional signature, and allow users to directly interact with the system by tweaking their tastes, which further contributes to both user experience and self-actualization. This work has practical implications for designing adaptive experiences. Explanation solutions in adaptive experiences might not always lead to a positive user experience, it highly depends on the application domain and the context (as studied in all four studies); it is essential to carefully investigate a specific explanation solution in combination with other design elements in different fields. Introducing control by allowing for direct interactivity (vs. indirect interactivity) in adaptive systems and providing feedback to users\u27 input by integrating their input into the algorithms would create a more engaging and interactive user experience (as studied in Study I and IV). And cumulatively, appropriate direct interaction with the system along with deliberate and thoughtful designs of explanation (including visualization design with the application environment fully considered), which are able to arouse user reflection or resonance, would potentially promote both user experience and user self-actualization

A Computational Framework for Planning Therapeutical Sessions aimed to Support the Prevention and Treatment of Mental Health Disorders using Emotional Virtual Agents

[EN] Interaction is defined as the realization of a reciprocal action between two or more people or things. Particularly in computer science, the term interaction refers to the discipline that studies the exchange of information between people and computers, and is generally known by the term Human-Computer Interaction (HCI). Good design decisions and an adequate development of the software is required for efficient HCI to facilitate the acceptability of computer-based applications by the users. In clinical settings it is essential to eliminate any barrier and facilitate the interaction between patients and the system. A smooth communication between the user and the computer-based application is fundamental to maximise the advantages and functionalities offered by the system. The design of these applications must consider the personal and current needs of the user by applying a User-Centered Design methodology. The main purpose of this research work is to contribute in the improvement of HCI-based applications addressed to the clinical context, particularly to enhance computer-based interactive sessions to support people suffering from a mental disorder such as Major Depression (MD). Thanks to the advances in Artificial Intelligence techniques, it is now possible to partially automate complex tasks such as the continuous provision of Cognitive-Behavioural Therapies (CBTs) to patients. These CBTs require good levels of adaptability and variability during the interaction with the patient that facilitates the acceptability in the user, an optimal usability and good level of engagement for a successful mid/long term use of the application and treatment adherence. The modelling of complex deliberative and affective processes in artificial systems can be applied to support the prevention and treatment of mental health related issues, enhancing the continuous and remote assistance of patients, saving some economical and clinical resources and reducing the waiting lists in the health services. In this regard, the efforts of this Thesis have been concentrated on the research of two main lines: (1) the generation and planning of adequate contents in an interactive system to support the prevention and treatment of MD based on characteristics of the user; and (2) the modelling of relevant affective processes able to communicate the contents in an emotional effective way taking into account the importance of the affective conditions associated with the MD in the users. Rule Based Systems and the appraisal theory of emotions have been the roots used to develop the main two modules of the computational Framework presented: the Contents Management and the Emotional Modules. Finally, the obtained Framework was integrated into two interactive systems to evaluate the achievement of the research objectives. The first system has been developed in the context of the Help4Mood European research project and its main aim was to support the remote treatment of patients with MD. The second scenario was a system developed to prevent MD and suicidal thoughts in the University community, which was developed in the context of the local PrevenDep research project. These evaluations have indicated that the proposed Framework has reached good levels of usability and acceptability in the target users thanks to the personalizations and adaptation capabilities of the contents and in the way how these contents are communicated to the user. The research work and the obtained results in this Thesis has contributed to the state of the art in HCI-based systems used as support in therapeutic interventions for the prevention and treatment of MD. This was obtained by the combination of a personalized content management to the patient, and the management of the affective processes associated to these pathologies. The developed work also identifies some research lines that need to be addressed in future works to get better HCI systems used for therapeutic purposes.[ES] Interactuar se define como la realización de una acción recíproca entre dos o más personas o cosas. Particularmente en informática, el término interacción se refiere a la disciplina que estudia el intercambio de información entre las personas y computadoras, y suele conocerse por el término anglosajón Human-Computer Interaction (HCI). Un buen diseño y un adecuado desarrollo del software es necesario para lograr una HCI eficiente que facilite la aceptabilidad del sistema por el usuario. En entornos clínicos es fundamental eliminar cualquier tipo de barrera y facilitar la interacción entre los pacientes y el computador. Es de vital importancia que haya una buena comunicación entre usuario y computador, por este motivo el sistema debe de estar diseñado pensando en las necesidades actuales, cambiantes y personales del usuario, basándose en la metodología de diseño centrado en el usuario. El propósito principal de esta investigación es la identificación de mejoras en HCI aplicada en entornos clínicos, en concreto para dar soporte a personas con trastornos mentales como la Depresión Mayor (DM) y que precisan de terapias psicológicas adecuadas y continuas. Gracias a técnicas de Inteligencia Artificial, es posible automatizar eficientemente ciertas acciones asociadas a los procesos de las terapias cognitivo-conductuales (CBTs, del inglés Cognitive-Behavioural Therapies). Los sistemas de ayuda a la CBT, requieren de una adaptabilidad y variabilidad en la interacción para favorecer la usabilidad del sistema y asegurar la continuidad de la motivación del paciente. Una buena gestión de esta automatización influiría en la aceptabilidad de los pacientes y podría mejorar su adherencia a los tratamientos y por consiguiente mejorar su estado de salud. Adicionalmente, la unión de procesos deliberativos dinámicos pueden liberar recursos clínicos, mejorando el control de los pacientes, y reduciendo los tiempos de espera y los costes económicos. En este sentido, los esfuerzos de esta Tesis se han centrado en la investigación de dos líneas diferentes: (1) la selección y planificación adecuada de los contenidos presentados durante la interacción a través de una planificación dinámica y personalizada, y (2) la adecuación de la comunicación de los contenidos hacia el paciente tomando en cuenta la importancia de los procesos afectivos asociados a estas patologías. Los Sistemas Basados en Reglas (SBR) han sido la herramienta utilizada para dar soporte a los dos módulos principales que componen el Framework presentado en esta Tesis: el módulo de gestión de los contenidos y el módulo emocional. Concluida la fase de diseño, desarrollo y testeo, el Framework fue adaptado e integrado en sistemas reales, para validar la viabilidad y la adecuación del marco de trabajo de esta Tesis. En primer lugar, el sistema se aplicó durante tres años en el tratamiento de la DM en varios centros clínicos europeos en el contexto del Proyecto Europeo de investigación Help4Mood. Finalmente, el sistema fue evaluado en la tarea de prevención de la DM y del suicidio en el Proyecto Local de investigación PrevenDep, de un año de duración. El feedback de estas evaluaciones demostraron que el HCI del Framework tiene unos niveles altos de usabilidad y aceptación, gracias a la personalización, variabilidad y adaptación de los contenidos y de la comunicación de los mismos. Los experimentos computacionales llevados a cabo en esta Tesis han permitido avanzar el estado del arte de sistemas computacionales emocionales aplicados en entornos terapéuticos para la prevención y tratamiento de la DM. Principalmente, gracias a la combinación de una gestión personalizada de los contenidos hacia el paciente tomando en cuenta la importancia de los procesos afectivos asociados a estas patologías. Este trabajo abre nuevas líneas de investigación, como la aplicación de este sistema en otras patologías de salud mental en las qu[CA] Interactuar es defineix com la realització d'una acció recíproca entre dos o més persones o coses. Particularment en informàtica, el terme interacció es refereix a la disciplina que estudia l'intercanvi d'informació entre les persones i computadores, i es sol conèixer pel terme anglosaxó Human-Computer Interaction (HCI). Un bon disseny i un adequat desenvolupament del software és necessari per aconseguir una HCI eficient que faciliti l'acceptabilitat del sistema per l'usuari. En entorns clínics és fonamental eliminar qualsevol tipus de barrera i facilitar la interacció entre els pacients i el computador. És de vital importància que hi hagi una bona comunicació entre l'usuari (o pacient) i el computador, per aquest motiu el sistema ha d'estar dissenyat pensant en les necessitats actuals, cambiants i personals de l'usuari, basant-se en la metodologia de disseny centrat en l'usuari. El propòsit principal d'aquesta investigació és la identificació de millores en HCI aplicada en entorns clínics, en concret per donar suport a persones amb trastorns mentals com la Depressió Major (DM) i que precisen de teràpies psicològiques adequades i contínues. Gràcies a tècniques d'Intel·ligència Artificial, és possible automatitzar eficientment certes accions asociades al processos de les teràpies cognitiu-conductuals. Els sistemes computacionals de ajuda a la CBT, requereixen d'una adaptabilitat i variabilitat en la interacció per afavorir la usabilitat del sistema i assegurar la continuïtat de la motiviació del pacient. Una bona gestió d'aquesta automatització influiria en l'acceptabilitat dels pacients i podria millorar la seva adherència als tractaments i per tant millorar el seu estat de salut. Addicionalment, la unió de processos deliberatius dinàmics poden alliberar recursos clínics, millorant el control dels pacients, i reduint els temps d'espera i els costos econòmics. En aquest sentit, els esforços d'aquesta Tesi s'han centrat en la investigació de dues línies diferents: (1) la selecció i planificació adequada dels continguts presentats durant la interacció a través d'una planificació dinàmica i personalitzada, i (2) l'adequació de la comunicació dels continguts cap al pacient tenint en compte la importància dels processos afectius associats a aquestes patologies. Els Sistemes Basats en Regles (SBR) han estat la eina utilitzada per donar suport als dos mòduls principals que componen el Framework presentat en aquesta Tesi: el mòdul de gestió dels continguts oferits a l'usuari; i el mòdul emocional. Conclosa la fase de disseny, desenvolupament i testeig, el Framework va ser adaptat als dominis corresponents i integrat en sistemes madurs per ser avaluat en dos escenaris reals, per validar la viabilitat i l'adequació del Framework d'aquesta tesi. Primerament, el sistema es va aplicar durant tres anys en el tractament de la DM major en diversos centres clínics europeus en el context del Projecte Europeu d'investigació Help4Mood. Finalment, el sistema va ser avaluat en la tasca de prevenció de la DM i del suïcidi al Projecte Local d'investigació PrevenDep, d'un any de durada. El feedback de les avaluacions han demostrat que el HCI del Framework obté uns nivells alts d'usabilitat i acceptació, gràcies a la personalització, variabilitat i adaptació dels continguts i de la comunicació. Els experiments computacionals duts a terme en aquesta Tesi han permès avançar l'estat de l'art de sistemes computacionals emocionals aplicats en entorns terapèutics per a la prevenció i tractament de la DM. Principalment, gracies a la combinació d'una gestió personalitzada dels continguts cap al pacient tenint en compte la importància dels processos afectius associats a aquestes patologies. Aquest treball obre noves línies d'investigació, com l'aplicació d'aquest sistema en altres patologies de salut mental en què sigui recomanable l'aplicació de sessions terapèutiques.Bresó Guardado, A. (2016). A Computational Framework for Planning Therapeutical Sessions aimed to Support the Prevention and Treatment of Mental Health Disorders using Emotional Virtual Agents [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/64082TESI

Agents for educational games and simulations

This book consists mainly of revised papers that were presented at the Agents for Educational Games and Simulation (AEGS) workshop held on May 2, 2011, as part of the Autonomous Agents and MultiAgent Systems (AAMAS) conference in Taipei, Taiwan. The 12 full papers presented were carefully reviewed and selected from various submissions. The papers are organized topical sections on middleware applications, dialogues and learning, adaption and convergence, and agent applications

Enhancing fan experience during live sports broadcasts through second screen applications

When sports fans attend live sports events, they usually engage in social experiences with friends, family members and other fans at the venue sharing the same affiliation. However, fans watching the same event through a live television broadcast end up not feeling so emotionally connected with the athletes and other fans as they would if they were watching it live, together with thousands of other fans. With this in mind, we seek to create mobile applications that deliver engaging social experiences involving remote fans watching live broadcasted sports events. Taking into account the growing use of mobile devices when watching TV broadcasts, these mobile applications explore the second screen concept, which allows users to interact with content that complements the TV broadcast. Within this context, we present a set of second screen application prototypes developed to test our concepts, the corresponding user studies and results, as well as suggestions on how to apply the prototypes’ concepts not only in different sports, but also during TV shows and electronic sports. Finally, we also present the challenges we faced and the guidelines we followed during the development and evaluation phases, which may give a considerable contribution to the development of future second screen applications for live broadcasted events

Player agency in interactive narrative: audience, actor & author

The question motivating this review paper is, how can computer-based interactive narrative be used as a constructivist learn- ing activity? The paper proposes that player agency can be used to link interactive narrative to learner agency in constructivist theory, and to classify approaches to interactive narrative. The traditional question driving research in interactive narrative is, ‘how can an in- teractive narrative deal with a high degree of player agency, while maintaining a coherent and well-formed narrative?’ This question derives from an Aristotelian approach to interactive narrative that, as the question shows, is inherently antagonistic to player agency. Within this approach, player agency must be restricted and manip- ulated to maintain the narrative. Two alternative approaches based on Brecht’s Epic Theatre and Boal’s Theatre of the Oppressed are reviewed. If a Boalian approach to interactive narrative is taken the conflict between narrative and player agency dissolves. The question that emerges from this approach is quite different from the traditional question above, and presents a more useful approach to applying in- teractive narrative as a constructivist learning activity