Probabilistic Perspectives on Collecting Human Uncertainty in Predictive Data Mining

In many areas of data mining, data is collected from humans beings. In this contribution, we ask the question of how people actually respond to ordinal scales. The main problem observed is that users tend to be volatile in their choices, i.e. complex cognitions do not always lead to the same decisions, but to distributions of possible decision outputs. This human uncertainty may sometimes have quite an impact on common data mining approaches and thus, the question of effective modelling this so called human uncertainty emerges naturally. Our contribution introduces two different approaches for modelling the human uncertainty of user responses. In doing so, we develop techniques in order to measure this uncertainty at the level of user inputs as well as the level of user cognition. With support of comprehensive user experiments and large-scale simulations, we systematically compare both methodologies along with their implications for personalisation approaches. Our findings demonstrate that significant amounts of users do submit something completely different (action) than they really have in mind (cognition). Moreover, we demonstrate that statistically sound evidence with respect to algorithm assessment becomes quite hard to realise, especially when explicit rankings shall be built

Affect-based information retrieval

One of the main challenges Information Retrieval (IR) systems face nowadays originates from the semantic gap problem: the semantic difference between a user’s query representation and the internal representation of an information item in a collection. The gap is further widened when the user is driven by an ill-defined information need, often the result of an anomaly in his/her current state of knowledge. The formulated search queries, which are submitted to the retrieval systems to locate relevant items, produce poor results that do not address the users’ information needs. To deal with information need uncertainty IR systems have employed in the past a range of feedback techniques, which vary from explicit to implicit. The first category of feedback techniques necessitates the communication of explicit relevance judgments, in return for better query reformulations and recommendations of relevant results. However, the latter happens at the expense of users’ cognitive resources and, furthermore, introduces an additional layer of complexity to the search process. On the other hand, implicit feedback techniques make inferences on what is relevant based on observations of user search behaviour. By doing so, they disengage users from the cognitive burden of document rating and relevance assessments. However, both categories of RF techniques determine topical relevance with respect to the cognitive and situational levels of interaction, failing to acknowledge the importance of emotions in cognition and decision making. In this thesis I investigate the role of emotions in the information seeking process and develop affective feedback techniques for interactive IR. This novel feedback framework aims to aid the search process and facilitate a more natural and meaningful interaction. I develop affective models that determine topical relevance based on information gathered from various sensory channels, and enhance their performance using personalisation techniques. Furthermore, I present an operational video retrieval system that employs affective feedback to enrich user profiles and offers meaningful recommendations of unseen videos. The use of affective feedback as a surrogate for the information need is formalised as the Affective Model of Browsing. This is a cognitive model that motivates the use of evidence extracted from the psycho-somatic mobilisation that occurs during cognitive appraisal. Finally, I address some of the ethical and privacy issues that arise from the social-emotional interaction between users and computer systems. This study involves questionnaire data gathered over three user studies, from 74 participants of different educational background, ethnicity and search experience. The results show that affective feedback is a promising area of research and it can improve many aspects of the information seeking process, such as indexing, ranking and recommendation. Eventually, it may be that relevance inferences obtained from affective models will provide a more robust and personalised form of feedback, which will allow us to deal more effectively with issues such as the semantic gap

Motivation Modelling and Computation for Personalised Learning of People with Dyslexia

The increasing development of e-learning systems in recent decades has benefited ubiquitous computing and education by providing freedom of choice to satisfy various needs and preferences about learning places and paces. Automatic recognition of learners’ states is necessary for personalised services or intervention to be provided in e-learning environments. In current literature, assessment of learners’ motivation for personalised learning based on the motivational states is lacking. An effective learning environment needs to address learners’ motivational needs, particularly, for those with dyslexia. Dyslexia or other learning difficulties can cause young people not to engage fully with the education system or to drop out due to complex reasons: in addition to the learning difficulties related to reading, writing or spelling, psychological difficulties are more likely to be ignored such as lower academic self-worth and lack of learning motivation caused by the unavoidable learning difficulties. Associated with both cognitive processes and emotional states, motivation is a multi-facet concept that consequences in the continued intention to use an e-learning system and thus a better chance of learning effectiveness and success. It consists of factors from intrinsic motivation driven by learners’ inner feeling of interest or challenges and those from extrinsic motivation associated with external reward or compliments. These factors represent learners’ various motivational needs; thus, understanding this requires a multidisciplinary approach. Combining different perspectives of knowledge on psychological theories and technology acceptance models with the empirical findings from a qualitative study with dyslexic students conducted in the present research project, motivation modelling for people with dyslexia using a hybrid approach is the main focus of this thesis. Specifically, in addition to the contribution to the qualitative conceptual motivation model and ontology-based computational model that formally expresses the motivational factors affecting users’ continued intention to use e-learning systems, this thesis also conceives a quantitative approach to motivation modelling. A multi-item motivation questionnaire is designed and employed in a quantitative study with dyslexic students, and structural equation modelling techniques are used to quantify the influences of the motivational factors on continued use intention and their interrelationships in the model. In addition to the traditional approach to motivation computation that relies on learners’ self-reported data, this thesis also employs dynamic sensor data and develops classification models using logistic regression for real-time assessment of motivational states. The rule-based reasoning mechanism for personalising motivational strategies and a framework of motivationally personalised e-learning systems are introduced to apply the research findings to e-learning systems in real-world scenarios. The motivation model, sensor-based computation and rule-based personalisation have been applied to a practical scenario with an essential part incorporated in the prototype of a gaze-based learning application that can output personalised motivational strategies during the learning process according to the real-time assessment of learners’ motivational states based on both the eye-tracking data in addition to users’ self-reported data. Evaluation results have indicated the advantage of the application implemented compared to the traditional one without incorporating the present research findings for monitoring learners’ motivation states with gaze data and generating personalised feedback. In summary, the present research project has: 1) developed a conceptual motivation model for students with dyslexia defining the motivational factors that influence their continued intention to use e-learning systems based on both a qualitative empirical study and prior research and theories; 2) developed an ontology-based motivation model in which user profiles, factors in the motivation model and personalisation options are structured as a hierarchy of classes; 3) designed a multi-item questionnaire, conducted a quantitative empirical study, used structural equation modelling to further explore and confirm the quantified impacts of motivational factors on continued use intention and the quantified relationships between the factors; 4) conducted an experiment to exploit sensors for motivation computation, and developed classification models for real-time assessment of the motivational states pertaining to each factor in the motivation model based on empirical sensor data including eye gaze data and EEG data; 5) proposed a sensor-based motivation assessment system architecture with emphasis on the use of ontologies for a computational representation of the sensor features used for motivation assessment in addition to the representation of the motivation model, and described the semantic rule-based personalisation of motivational strategies; 6) proposed a framework of motivationally personalised e-learning systems based on the present research, with the prototype of a gaze-based learning application designed, implemented and evaluated to guide future work

Novel Datasets, User Interfaces and Learner Models to Improve Learner Engagement Prediction on Educational Videos

With the emergence of Open Education Resources (OERs), educational content creation has rapidly scaled up, making a large collection of new materials made available. Among these, we find educational videos, the most popular modality for transferring knowledge in the technology-enhanced learning paradigm. Rapid creation of learning resources opens up opportunities in facilitating sustainable education, as the potential to personalise and recommend specific materials that align with individual users’ interests, goals, knowledge level, language and stylistic preferences increases. However, the quality and topical coverage of these materials could vary significantly, posing significant challenges in managing this large collection, including the risk of negative user experience and engagement with these materials. The scarcity of support resources such as public datasets is another challenge that slows down the development of tools in this research area. This thesis develops a set of novel tools that improve the recommendation of educational videos. Two novel datasets and an e-learning platform with a novel user interface are developed to support the offline and online testing of recommendation models for educational videos. Furthermore, a set of learner models that accounts for the learner interests, knowledge, novelty and popularity of content is developed through this thesis. The different models are integrated together to propose a novel learner model that accounts for the different factors simultaneously. The user studies conducted on the novel user interface show that the new interface encourages users to explore the topical content more rigorously before making relevance judgements about educational videos. Offline experiments on the newly constructed datasets show that the newly proposed learner models outperform their relevant baselines significantly

Layered evaluation of interactive adaptive systems : framework and formative methods

Peer reviewedPostprin

CHORUS Deliverable 2.1: State of the Art on Multimedia Search Engines

Based on the information provided by European projects and national initiatives related to multimedia search as well as domains experts that participated in the CHORUS Think-thanks and workshops, this document reports on the state of the art related to multimedia content search from, a technical, and socio-economic perspective. The technical perspective includes an up to date view on content based indexing and retrieval technologies, multimedia search in the context of mobile devices and peer-to-peer networks, and an overview of current evaluation and benchmark inititiatives to measure the performance of multimedia search engines. From a socio-economic perspective we inventorize the impact and legal consequences of these technical advances and point out future directions of research