The Recommendation Dashboard: A System to Visualise and Organise Recommendations

Abstract Recommender systems are becoming common tools supporting automatic, context-based retrieval of resources. When the number of retrieved resources grows large visual tools are required that leverage the capacity of human vision to analyse large amounts of information. We introduce a Web-based visual tool for exploring and organising recommendations retrieved from multiple sources along dimensions relevant to cultural heritage and educational context. Our tool provides several views supporting filtering in the result set and integrates a bookmarking system for organising relevant resources into topic collections. Building upon these features we envision a system which derives user's interests from performed actions and uses this information to support the recommendation process. We also report on results of the performed usability evaluation and derive directions for further development

Establishing User Requirements for a Recommender System in an Online Union Catalogue: an Investigation of WorldCat.org

This project, undertaken in collaboration with OCLC, aimed to investigate the potential role of recommendations within WorldCat, the publicly accessible union catalogue of libraries participating in the OCLC global cooperative. The goal of the project was a set of conceptual design guidelines for a WorldCat.org recommender system, based on a comprehensive understanding of the systems users and their needs. Taking a mixed-methods approach, the investigation consisted of four phases. Phase one consisted of twenty-one focus groups with key user goups held in three locations; the UK, the US, and Australia and New Zealand. Phase 2 consisted of a pop-up survey implemented on WorldCat.org, and gathered 2,918 responses. Phase three represented an analysis of two months of WorldCat.org transaction log data, consisting of over 15,000,000 sessions. Phase four was a lab based user study investigating and comparing the use of WorldCat.org with Amazon. Findings from each strand were integrated, and the key themes to emerge from the research are discussed. Different methods of classifying the WorldCat.org user population are presented, along with a taxonomy of work- and search-tasks. Key perspectives on the utility of a recommender system are considered, along with a reflection on how the information search behaviour exhibited by users interacting with recommendations while undertaking typical catalogue tasks can be interpreted. Based on the enriched perspective of the system, and the role of recommendation in the catalogue, a series of conceptual design specifications are presented for the development of a WorldCat.org recommender system

Information Management for Digital Learners : Introduction, Challenges, and Concepts of Personal Information Management for Individual Learners

The current cultural transition of our society into a digital society influences all aspects of human life. New technologies like the Internet and mobile devices enable an unobstructed access to knowledge in worldwide networks. These advancements bring with them a great freedom in decisions and actions of individuals but also a growing demand for an appropriate mastering of this freedom of choice and the amount of knowledge that has become available today. Naturally, this observable rise and progress of new technologies—gently but emphatically becoming part of people’s everyday lives—not only changes the way people work, communicate, and shape their leisure but also the way people learn. This thesis is dedicated to an examination of how learners can meet these requirements with the support that modern technology is able to provide to learners. More precisely, this thesis places a particular emphasis that is absent from previous work in the field and thus makes it distinctive: the explicit focus on individual learners. As a result, the main concern of this thesis can be described as the examination, development, and implementation of personal information management in learning. Altogether two different steps towards a solution have been chosen: the development of a theoretical framework and its practical implementation into a comprehensive concept. To establish a theoretical framework for personal information management in learning, the spheres of learning, e-learning, and personalised learning have been combined with theories of organisational and personal knowledge management to form a so far unique holistic view of personal information management in learning. The development of this framework involves the identification of characteristics, needs, and challenges that distinguish individual learners from within the larger crowd of uniform learners. The theoretical framework defined within the first part is transferred to a comprehensive technical concept for personal information management in learning. The realisation and design of this concept as well as its practical implementation are strongly characterised by the utilisation of information retrieval techniques to support individual learners. The characteristic feature of the resulting system is a flexible architecture that enables the unified acquisition, representation, and organisation of information related to an individual’s learning and supports an improved find-ability of personal information across all relevant sources of information. The most important results of this thesis have been validated by a comparison with current projects in related areas and within a user study.Der gegenwärtige Wandel unserer Gesellschaft zu einer digitalen Gesellschaft hat weitreichenden Einfluss auf alle Aspekte des menschlichen Lebens. Neue Technologien wie das Internet und mobile Geräte zur Nutzung dieser Technologien ermöglichen einen nahezu ungehinderten Zugriff auf Wissen in weltweiten Netzwerken. Dieser Fortschritt bringt einerseits einen großen Freiheitsgrad für Entscheidungen und Handlungen des Einzelnen, andererseits aber auch eine immer lauter werdende Forderung nach Strategien für einen adäquaten Umgang mit dieser Freiheit und der verfügbaren Menge an Informationen. Naturgemäß verändern dieser Fortschritt und die zugehörigen Technologien nicht nur unser Arbeitsleben und den privaten Alltag, sondern auch die Art und Weise zu lernen. Diese Arbeit beschäftigt sich mit der Frage, wie Lernende diesen neuen Anforderungen gerecht werden und mithilfe von modernen Technologien in einem adäquaten Informationsmanagement unterstützt werden können. Die Besonderheit liegt dabei in einem ausschließlichen Fokus individuell Lernender, genauer gesagt jenen, die sich eigenständig auf individuellen Lernpfaden bewegen. Zusammengefasst untersucht diese Arbeit daher Möglichkeiten des personalisierten Informationsmanagements für Lernende. Die Untersuchung dieser Fragestellung erfolgt auf zwei Ebenen. Die erste Ebene dieser Arbeit umfasst eine theoretische Untersuchung der Thematik. Zu diesem Zweck wird ein übergreifendes Rahmenwerk für das persönliche Informationsmanagement von Lernenden entwickelt, das eine ganzheitliche Betrachtung dieser Fragestellung ermöglicht. Das entwickelte Rahmenwerk zeichnet sich insbesondere durch eine Verschmelzung der Domänen E-Learning und Wissensmanagement aus. Dazu werden im Rahmen dieser theoretischen Untersuchung prägende Facetten des Lernens beschrieben und Theorien des organisatorischen Wissensmanagements zur Bewältigung des persönlichen Informationsmanagements untersucht. Dies führt schließlich zu einer Charakterisierung von individuell Lernenden, der Identifikation grundlegender Herausforderungen für diese Lernenden sowie einem Modell zur Beschreibung des individuellen Informations- und Wissensmanagements. Die zweite Ebene dieser Arbeit umfasst die Umsetzung des entwickelten Rahmenwerks in ein praktisches Konzept zur effizienten Verwaltung von persönlichen Lerninhalten und -informationen einzelner Lernender. Das realisierte System ist dabei durch die Berücksichtigung von Informationsbedürfnissen individuell Lernender sowie besonders durch den gezielten Einsatz von Information Retrieval Techniken zur Unterstützung dieser Lernenden gekennzeichnet. Das konstituierende Merkmal dieses Systems ist daher eine flexible Architektur, die die Erfassung von Lernobjekten unter besonderer Berücksichtigung des Lernkontexts erlaubt. Detaillierter betrachtet ermöglicht die Erfassung von Basisinformation in Form von Lernobjekten in Kombination mit hierarchischen und nicht-hierarchischen Zusatzinformationen eine individuelle und umfassende Verwaltung von Lerninhalten und -informationen, die auch eine verbesserte Wiederauffindbarkeit dieser Informationen zu einem späteren Zeitpunkt unterstützt. Die wichtigsten Ergebnisse dieser Arbeit werden aktuellen Entwicklungen und Projekten in verwandten Bereichen gegenübergestellt und im Rahmen einer Nutzerstudie grundlegend validiert

A Usability Approach to Improving the User Experience in Web Directories

Submitted for the degree of Doctor of Philosophy, Queen Mary, University of Londo

A usability approach to improving the user experience in web directories

PhDWeb directories are hierarchically organised website collections that offer users subjectbased access to the Web. They played a significant part in navigating the Web in the past but their role has been weakened in recent years due to their cumbersome expanding collections. This thesis presents a unified framework combining the advantages of personalisation and redefined directory search for improving the usability of Web directories. The thesis begins with an examination of classification schemes that identifies the rigidity of hierarchical classifications and their suitability for Web directories in contrast to faceted classifications. This leads on to an Ontological Sketch Modelling (OSM) case study which identifies the misfits affecting user navigation in Web directories from known rigidity issues. The thesis continues with a review of personalisation techniques and a discussion of the user search model of Web directories following the suggested directions of improvement from the case study. A proposed user-centred framework to improve the usability of Web directories which consists of an individual content-based personalisation model and a redefined search model is then implemented as D-Persona and D-Search respectively. The remainder of the thesis is concerned with a usability test of D-Persona and D-Search aimed at discovering the efficiency, effectiveness and user satisfaction of the solution. This involves an experimental design, test results and discussions for the comparative user study. This thesis extracts a formal definition of the rigidity of hierarchies from their characteristics and justifies why hierarchies are still better suited than facets in organising Web directories. Second, it identifies misfits causing poor usability in Web directories based on the discovered rigidity of hierarchies. Third, it proposes a solution to tackle the misfits and improve the usability of Web directories which has been experimentally proved to be successful

Decision Support Elements and Enabling Techniques to Achieve a Cyber Defence Situational Awareness Capability

[ES] La presente tesis doctoral realiza un análisis en detalle de los elementos de decisión necesarios para mejorar la comprensión de la situación en ciberdefensa con especial énfasis en la percepción y comprensión del analista de un centro de operaciones de ciberseguridad (SOC). Se proponen dos arquitecturas diferentes basadas en el análisis forense de flujos de datos (NF3). La primera arquitectura emplea técnicas de Ensemble Machine Learning mientras que la segunda es una variante de Machine Learning de mayor complejidad algorítmica (lambda-NF3) que ofrece un marco de defensa de mayor robustez frente a ataques adversarios. Ambas propuestas buscan automatizar de forma efectiva la detección de malware y su posterior gestión de incidentes mostrando unos resultados satisfactorios en aproximar lo que se ha denominado un SOC de próxima generación y de computación cognitiva (NGC2SOC). La supervisión y monitorización de eventos para la protección de las redes informáticas de una organización debe ir acompañada de técnicas de visualización. En este caso, la tesis aborda la generación de representaciones tridimensionales basadas en métricas orientadas a la misión y procedimientos que usan un sistema experto basado en lógica difusa. Precisamente, el estado del arte muestra serias deficiencias a la hora de implementar soluciones de ciberdefensa que reflejen la relevancia de la misión, los recursos y cometidos de una organización para una decisión mejor informada. El trabajo de investigación proporciona finalmente dos áreas claves para mejorar la toma de decisiones en ciberdefensa: un marco sólido y completo de verificación y validación para evaluar parámetros de soluciones y la elaboración de un conjunto de datos sintéticos que referencian unívocamente las fases de un ciberataque con los estándares Cyber Kill Chain y MITRE ATT & CK.[CA] La present tesi doctoral realitza una anàlisi detalladament dels elements de decisió necessaris per a millorar la comprensió de la situació en ciberdefensa amb especial èmfasi en la percepció i comprensió de l'analista d'un centre d'operacions de ciberseguretat (SOC). Es proposen dues arquitectures diferents basades en l'anàlisi forense de fluxos de dades (NF3). La primera arquitectura empra tècniques de Ensemble Machine Learning mentre que la segona és una variant de Machine Learning de major complexitat algorítmica (lambda-NF3) que ofereix un marc de defensa de major robustesa enfront d'atacs adversaris. Totes dues propostes busquen automatitzar de manera efectiva la detecció de malware i la seua posterior gestió d'incidents mostrant uns resultats satisfactoris a aproximar el que s'ha denominat un SOC de pròxima generació i de computació cognitiva (NGC2SOC). La supervisió i monitoratge d'esdeveniments per a la protecció de les xarxes informàtiques d'una organització ha d'anar acompanyada de tècniques de visualització. En aquest cas, la tesi aborda la generació de representacions tridimensionals basades en mètriques orientades a la missió i procediments que usen un sistema expert basat en lògica difusa. Precisament, l'estat de l'art mostra serioses deficiències a l'hora d'implementar solucions de ciberdefensa que reflectisquen la rellevància de la missió, els recursos i comeses d'una organització per a una decisió més ben informada. El treball de recerca proporciona finalment dues àrees claus per a millorar la presa de decisions en ciberdefensa: un marc sòlid i complet de verificació i validació per a avaluar paràmetres de solucions i l'elaboració d'un conjunt de dades sintètiques que referencien unívocament les fases d'un ciberatac amb els estàndards Cyber Kill Chain i MITRE ATT & CK.[EN] This doctoral thesis performs a detailed analysis of the decision elements necessary to improve the cyber defence situation awareness with a special emphasis on the perception and understanding of the analyst of a cybersecurity operations center (SOC). Two different architectures based on the network flow forensics of data streams (NF3) are proposed. The first architecture uses Ensemble Machine Learning techniques while the second is a variant of Machine Learning with greater algorithmic complexity (lambda-NF3) that offers a more robust defense framework against adversarial attacks. Both proposals seek to effectively automate the detection of malware and its subsequent incident management, showing satisfactory results in approximating what has been called a next generation cognitive computing SOC (NGC2SOC). The supervision and monitoring of events for the protection of an organisation's computer networks must be accompanied by visualisation techniques. In this case, the thesis addresses the representation of three-dimensional pictures based on mission oriented metrics and procedures that use an expert system based on fuzzy logic. Precisely, the state-of-the-art evidences serious deficiencies when it comes to implementing cyber defence solutions that consider the relevance of the mission, resources and tasks of an organisation for a better-informed decision. The research work finally provides two key areas to improve decision-making in cyber defence: a solid and complete verification and validation framework to evaluate solution parameters and the development of a synthetic dataset that univocally references the phases of a cyber-attack with the Cyber Kill Chain and MITRE ATT & CK standards.Llopis Sánchez, S. (2023). Decision Support Elements and Enabling Techniques to Achieve a Cyber Defence Situational Awareness Capability [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/19424

Collaborative Knowledge Visualisation for Cross-Community Knowledge Exchange

The notion of communities as informal social networks based on shared interests or common practices has been increasingly used as an important unit of analysis when considering the processes of cooperative creation and sharing of knowledge. While knowledge exchange within communities has been extensively researched, different studies observed the importance of cross-community knowledge exchange for the creation of new knowledge and innovation in knowledge-intensive organizations. Especially in knowledge management a critical problem has become the need to support the cooperation and exchange of knowledge between different communities with highly specialized expertise and activities. Though several studies discuss the importance and difficulties of knowledge sharing across community boundaries, the development of technological support incorporating these findings has been little addressed. This work presents an approach to supporting cross-community knowledge exchange based on using knowledge visualisation for facilitating information access in unfamiliar community domains. The theoretical grounding and practical relevance of the proposed approach are ensured by defining a requirements model that integrates theoretical frameworks for cross-community knowledge exchange with practical needs of typical knowledge management processes and sensemaking tasks in information access in unfamiliar domains. This synthesis suggests that visualising knowledge structures of communities and supporting the discovery of relationships between them during access to community spaces, could provide valuable support for cross-community discovery and sharing of knowledge. This is the main hypothesis investigated in this thesis. Accordingly, a novel method is developed for eliciting and visualising implicit knowledge structures of individuals and communities in form of dynamic knowledge maps that make the elicited knowledge usable for semantic exploration and navigation of community spaces. The method allows unobtrusive construction of personal and community knowledge maps based on user interaction with information and their use for dynamic classification of information from a specific point of view. The visualisation model combines Document Maps presenting main topics, document clusters and relationships between knowledge reflected in community spaces with Concept Maps visualising personal and shared conceptual structures of community members. The technical realization integrates Kohonen’s self-organizing maps with extraction of word categories from texts, collaborative indexing and personalised classification based on user-induced templates. This is accompanied by intuitive visualisation and interaction with complex information spaces based on multi-view navigation of document landscapes and concept networks. The developed method is prototypically implemented in form of an application framework, a concrete system and a visual information interface for multi-perspective access to community information spaces, the Knowledge Explorer. The application framework implements services for generating and using personal and community knowledge maps to support explicit and implicit knowledge exchange between members of different communities. The Knowledge Explorer allows simultaneous visualisation of different personal and community knowledge structures and enables their use for structuring, exploring and navigating community information spaces from different points of view. The empirical evaluation in a comparative laboratory study confirms the adequacy of the developed solutions with respect to specific requirements of the cross-community problem and demonstrates much better quality of knowledge access compared to a standard information seeking reference system. The developed evaluation framework and operative measures for quality of knowledge access in cross-community contexts also provide a theoretically grounded and practically feasible method for further developing and evaluating new solutions addressing this important but little investigated problem

A ranking framework and evaluation for diversity-based retrieval

There has been growing momentum in building information retrieval (IR) systems that consider both relevance and diversity of retrieved information, which together improve the usefulness of search results as perceived by users. Some users may genuinely require a set of multiple results to satisfy their information need as there is no single result that completely fulfils the need. Others may be uncertain about their information need and they may submit ambiguous or broad (faceted) queries, either intentionally or unintentionally. A sensible approach to tackle these problems is to diversify search results to address all possible senses underlying those queries or all possible answers satisfying the information need. In this thesis, we explore three aspects of diversity-based document retrieval: 1) recommender systems, 2) retrieval algorithms, and 3) evaluation measures. This first goal of this thesis is to provide an understanding of the need for diversity in search results from the users’ perspective. We develop an interactive recommender system for the purpose of a user study. Designed to facilitate users engaged in exploratory search, the system is featured with content-based browsing, aspectual interfaces, and diverse recommendations. While the diverse recommendations allow users to discover more and different aspects of a search topic, the aspectual interfaces allow users to manage and structure their own search process and results regarding aspects found during browsing. The recommendation feature mines implicit relevance feedback information extracted from a user’s browsing trails and diversifies recommended results with respect to document contents. The result of our user-centred experiment shows that result diversity is needed in realistic retrieval scenarios. Next, we propose a new ranking framework for promoting diversity in a ranked list. We combine two distinct result diversification patterns; this leads to a general framework that enables the development of a variety of ranking algorithms for diversifying documents. To validate our proposal and to gain more insights into approaches for diversifying documents, we empirically compare our integration framework against a common ranking approach (i.e. the probability ranking principle) as well as several diversity-based ranking strategies. These include maximal marginal relevance, modern portfolio theory, and sub-topic-aware diversification based on sub-topic modelling techniques, e.g. clustering, latent Dirichlet allocation, and probabilistic latent semantic analysis. Our findings show that the two diversification patterns can be employed together to improve the effectiveness of ranking diversification. Furthermore, we find that the effectiveness of our framework mainly depends on the effectiveness of the underlying sub-topic modelling techniques. Finally, we examine evaluation measures for diversity retrieval. We analytically identify an issue affecting the de-facto standard measure, novelty-biased discounted cumulative gain (α-nDCG). This issue prevents the measure from behaving as desired, i.e. assessing the effectiveness of systems that provide complete coverage of sub-topics by avoiding excessive redundancy. We show that this issue is of importance as it highly affects the evaluation of retrieval systems, specifically by overrating top-ranked systems that repeatedly retrieve redundant information. To overcome this issue, we derive a theoretically sound solution by defining a safe threshold on a query-basis. We examine the impact of arbitrary settings of the α-nDCG parameter. We evaluate the intuitiveness and reliability of α-nDCG when using our proposed setting on both real and synthetic rankings. We demonstrate that the diversity of document rankings can be intuitively measured by employing the safe threshold. Moreover, our proposal does not harm, but instead increases the reliability of the measure in terms of discriminative power, stability, and sensitivity