ELVIS: Entertainment-led video summaries

© ACM, 2010. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACM Transactions on Multimedia Computing, Communications, and Applications, 6(3): Article no. 17 (2010) http://doi.acm.org/10.1145/1823746.1823751Video summaries present the user with a condensed and succinct representation of the content of a video stream. Usually this is achieved by attaching degrees of importance to low-level image, audio and text features. However, video content elicits strong and measurable physiological responses in the user, which are potentially rich indicators of what video content is memorable to or emotionally engaging for an individual user. This article proposes a technique that exploits such physiological responses to a given video stream by a given user to produce Entertainment-Led VIdeo Summaries (ELVIS). ELVIS is made up of five analysis phases which correspond to the analyses of five physiological response measures: electro-dermal response (EDR), heart rate (HR), blood volume pulse (BVP), respiration rate (RR), and respiration amplitude (RA). Through these analyses, the temporal locations of the most entertaining video subsegments, as they occur within the video stream as a whole, are automatically identified. The effectiveness of the ELVIS technique is verified through a statistical analysis of data collected during a set of user trials. Our results show that ELVIS is more consistent than RANDOM, EDR, HR, BVP, RR and RA selections in identifying the most entertaining video subsegments for content in the comedy, horror/comedy, and horror genres. Subjective user reports also reveal that ELVIS video summaries are comparatively easy to understand, enjoyable, and informative

Analysing user physiological responses for affective video summarisation

This is the post-print version of the final paper published in Displays. The published article is available from the link below. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. Copyright @ 2009 Elsevier B.V.Video summarisation techniques aim to abstract the most significant content from a video stream. This is typically achieved by processing low-level image, audio and text features which are still quite disparate from the high-level semantics that end users identify with (the ‘semantic gap’). Physiological responses are potentially rich indicators of memorable or emotionally engaging video content for a given user. Consequently, we investigate whether they may serve as a suitable basis for a video summarisation technique by analysing a range of user physiological response measures, specifically electro-dermal response (EDR), respiration amplitude (RA), respiration rate (RR), blood volume pulse (BVP) and heart rate (HR), in response to a range of video content in a variety of genres including horror, comedy, drama, sci-fi and action. We present an analysis framework for processing the user responses to specific sub-segments within a video stream based on percent rank value normalisation. The application of the analysis framework reveals that users respond significantly to the most entertaining video sub-segments in a range of content domains. Specifically, horror content seems to elicit significant EDR, RA, RR and BVP responses, and comedy content elicits comparatively lower levels of EDR, but does seem to elicit significant RA, RR, BVP and HR responses. Drama content seems to elicit less significant physiological responses in general, and both sci-fi and action content seem to elicit significant EDR responses. We discuss the implications this may have for future affective video summarisation approaches

Novel Methods Using Human Emotion and Visual Features for Recommending Movies

Postponed access: the file will be accessible after 2022-06-01This master thesis investigates novel methods using human emotion as contextual information to estimate and elicit ratings when watching movie trailers. The aim is to acquire user preferences without the intrusive and time-consuming behavior of Explicit Feedback strategies, and generate quality recommendations. The proposed preference-elicitation technique is implemented as an Emotion-based Filtering technique (EF) to generate recommendations, and is evaluated against two other recommendation techniques. One Visual-based Filtering technique, using low-level visual features of movies, and one Collaborative Filtering (CF) using explicit ratings. In terms of \textit{Accuracy}, we found the Emotion-based Filtering technique (EF) to perform better than the two other filtering techniques. In terms of \textit{Diversity}, the Visual-based Filtering (VF) performed best. We further analyse the obtained data to see if movie genres tend to induce specific emotions, and the potential correlation between emotional responses of users and visual features of movie trailers. When investigating emotional responses, we found that \textit{joy} and \textit{disgust} tend to be more prominent in movie genres than other emotions. Our findings also suggest potential correlations on a per movie level. The proposed Visual-based Filtering technique can be adopted as an Implicit Feedback strategy to obtain user preferences. For future work, we will extend the experiment with more participants and build stronger affective profiles to be studied when recommending movies.Masteroppgave i informasjonsvitenskapINFO390MASV-INF

Intelligent and Energy-Efficient Data Prioritization in Green Smart Cities: Current Challenges and Future Directions

[EN] The excessive use of digital devices such as cameras and smartphones in smart cities has produced huge data repositories that require automatic tools for efficient browsing, searching, and management. Data prioritization (DP) is a technique that produces a condensed form of the original data by analyzing its contents. Current DP studies are either concerned with data collected through stable capturing devices or focused on prioritization of data of a certain type such as surveillance, sports, or industry. This necessitates the need for DP tools that intelligently and cost-effectively prioritize a large variety of data for detecting abnormal events and hence effectively manage them, thereby making the current smart cities greener. In this article, we first carry out an in-depth investigation of the recent approaches and trends of DP for data of different natures, genres, and domains of two decades in green smart cities. Next, we propose an energy-efficient DP framework by intelligent integration of the Internet of Things, artificial intelligence, and big data analytics. Experimental evaluation on real-world surveillance data verifies the energy efficiency and applicability of this framework in green smart cities. Finally, this article highlights the key challenges of DP, its future requirements, and propositions for integration into green smart citiesThis work was supported by a National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) (no. 2016R-1A2B4011712).Muhammad, K.; Lloret, J.; Baik, SW. (2019). Intelligent and Energy-Efficient Data Prioritization in Green Smart Cities: Current Challenges and Future Directions. IEEE Communications Magazine. 57(2):60-65. https://doi.org/10.1109/MCOM.2018.1800371S606557

Protocol for a randomized controlled trial examining multilevel prediction of response to behavioral activation and exposure-based therapy for generalized anxiety disorder.

BACKGROUND:Only 40-60% of patients with generalized anxiety disorder experience long-lasting improvement with gold standard psychosocial interventions. Identifying neurobehavioral factors that predict treatment success might provide specific targets for more individualized interventions, fostering more optimal outcomes and bringing us closer to the goal of "personalized medicine." Research suggests that reward and threat processing (approach/avoidance behavior) and cognitive control may be important for understanding anxiety and comorbid depressive disorders and may have relevance to treatment outcomes. This study was designed to determine whether approach-avoidance behaviors and associated neural responses moderate treatment response to exposure-based versus behavioral activation therapy for generalized anxiety disorder. METHODS/DESIGN:We are conducting a randomized controlled trial involving two 10-week group-based interventions: exposure-based therapy or behavioral activation therapy. These interventions focus on specific and unique aspects of threat and reward processing, respectively. Prior to and after treatment, participants are interviewed and undergo behavioral, biomarker, and neuroimaging assessments, with a focus on approach and avoidance processing and decision-making. Primary analyses will use mixed models to examine whether hypothesized approach, avoidance, and conflict arbitration behaviors and associated neural responses at baseline moderate symptom change with treatment, as assessed using the Generalized Anxiety Disorder-7 item scale. Exploratory analyses will examine additional potential treatment moderators and use data reduction and machine learning methods. DISCUSSION:This protocol provides a framework for how studies may be designed to move the field toward neuroscience-informed and personalized psychosocial treatments. The results of this trial will have implications for approach-avoidance processing in generalized anxiety disorder, relationships between levels of analysis (i.e., behavioral, neural), and predictors of behavioral therapy outcome. TRIAL REGISTRATION:The study was retrospectively registered within 21 days of first participant enrollment in accordance with FDAAA 801 with ClinicalTrials.gov, NCT02807480. Registered on June 21, 2016, before results

Making effective use of healthcare data using data-to-text technology

Healthcare organizations are in a continuous effort to improve health outcomes, reduce costs and enhance patient experience of care. Data is essential to measure and help achieving these improvements in healthcare delivery. Consequently, a data influx from various clinical, financial and operational sources is now overtaking healthcare organizations and their patients. The effective use of this data, however, is a major challenge. Clearly, text is an important medium to make data accessible. Financial reports are produced to assess healthcare organizations on some key performance indicators to steer their healthcare delivery. Similarly, at a clinical level, data on patient status is conveyed by means of textual descriptions to facilitate patient review, shift handover and care transitions. Likewise, patients are informed about data on their health status and treatments via text, in the form of reports or via ehealth platforms by their doctors. Unfortunately, such text is the outcome of a highly labour-intensive process if it is done by healthcare professionals. It is also prone to incompleteness, subjectivity and hard to scale up to different domains, wider audiences and varying communication purposes. Data-to-text is a recent breakthrough technology in artificial intelligence which automatically generates natural language in the form of text or speech from data. This chapter provides a survey of data-to-text technology, with a focus on how it can be deployed in a healthcare setting. It will (1) give an up-to-date synthesis of data-to-text approaches, (2) give a categorized overview of use cases in healthcare, (3) seek to make a strong case for evaluating and implementing data-to-text in a healthcare setting, and (4) highlight recent research challenges.Comment: 27 pages, 2 figures, book chapte

The use of Artificial intelligence in school science: a systematic literature review

Artificial Intelligence is widely used across contexts and for different purposes, including the field of education. However, a review of the literature showcases that while there exist various review studies on the use of AI in education, missing remains a review focusing on science education. To address this gap, we carried out a systematic literature review between 2010 and 2021, driven by three questions: a) What types of AI applications are used in school science? b) For what teaching content are AI applications in school science used? and, c) What is the impact of AI applications on teaching and learning of school science? The studies reviewed (n = 22) included nine different types of AI applications: automated assessment, automated feedback, learning analytics, adaptive learning systems, intelligent tutoring systems, multilabel text classification, chatbot, expert systems, and mind wandering detection. The majority of the AI applications are used in geoscience or physics and AI applications are used to support either knkowledge construction or skills development. In terms of the impact of AI applications, this is found across the following: learning achievement, argumentation skills, learning experience, and teaching. Missing remains an examination of learners’ and teachers’ experiences with the use of AI in school science, interdisciplinary approaches to AI implementation, as well as an examination of issues related to ethics and biase

New methods for collaborative experiential learning to provide personalised formative assessment

Supporting diverse and rapidly changing learning styles of new digital age generations is one of the major hurdles to higher education in the age of massification of education markets. Higher education institutions must now utilize unprecedented network speed and mobile technology to create stimulating learning environments for new digital age generations. This paper presents a new learning and teaching model that combines dynamic learning space (DLS) and mobile collaborative experimental learning (MCEL) for supporting diverse learning styles of students. DLS assists students with stateof-art modern wireless network technologies in order to support fast-paced, multi-tasking, data and content intensive collaborative learning in class. The model further extends student learning activities beyond classroom by allowing students to continue their learning anywhere and anytime conveniently using their mobile devices. MCEL provides automated continuous personalized formative-feedback 24/7. The main objectives of the model are to improve student engagement and to provide ownership of their learning journey, experiential learning, contextualized learning, and formative assessment at low cost. The model employs three factors that influence collaborative experiential learning and formative assessment. The three factors are: - The use of learning space within the classroom - Wireless learning technology - Mobile learning system (m-Learning) Pilot studies of the model are conducted and evaluated on two groups of postgraduate students. Their participation is observed, and a survey is conducted. The results show that (1) DLS encourages high-level learning and diverse learning styles to move away from passive low-level knowledge intensive learning activities; (2) MCEL supports Bigg's constructive alignment in curriculum design, contextualized experimental learning, and personalized formative learning