Knowledge aggregation in people recommender systems : matching skills to tasks

People recommender systems (PRS) are a special type of RS. They are often adopted to identify people capable of performing a task. Recommending people poses several challenges not exhibited in traditional RS. Elements such as availability, overload, unresponsiveness, and bad recommendations can have adverse effects. This thesis explores how people’s preferences can be elicited for single-event matchmaking under uncertainty and how to align them with appropriate tasks. Different methodologies are introduced to profile people, each based on the nature of the information from which it was obtained. These methodologies are developed into three use cases to illustrate the challenges of PRS and the steps taken to address them. Each one emphasizes the priorities of the matching process and the constraints under which these recommendations are made. First, multi-criteria profiles are derived completely from heterogeneous sources in an implicit manner characterizing users from multiple perspectives and multi-dimensional points-of-view without influence from the user. The profiles are introduced to the conference reviewer assignment problem. Attention is given to distribute people across items in order reduce potential overloading of a person, and neglect or rejection of a task. Second, people’s areas of interest are inferred from their resumes and expressed in terms of their uncertainty avoiding explicit elicitation from an individual or outsider. The profile is applied to a personnel selection problem where emphasis is placed on the preferences of the candidate leading to an asymmetric matching process. Third, profiles are created by integrating implicit information and explicitly stated attributes. A model is developed to classify citizens according to their lifestyles which maintains the original information in the data set throughout the cluster formation. These use cases serve as pilot tests for generalization to real-life implementations. Areas for future application are discussed from new perspectives.Els sistemes de recomanació de persones (PRS) són un tipus especial de sistemes recomanadors (RS). Sovint s’utilitzen per identificar persones per a realitzar una tasca. La recomanació de persones comporta diversos reptes no exposats en la RS tradicional. Elements com la disponibilitat, la sobrecàrrega, la falta de resposta i les recomanacions incorrectes poden tenir efectes adversos. En aquesta tesi s'explora com es poden obtenir les preferències dels usuaris per a la definició d'assignacions sota incertesa i com aquestes assignacions es poden alinear amb tasques definides. S'introdueixen diferents metodologies per definir el perfil d’usuaris, cadascun en funció de la naturalesa de la informació necessària. Aquestes metodologies es desenvolupen i s’apliquen en tres casos d’ús per il·lustrar els reptes dels PRS i els passos realitzats per abordar-los. Cadascun destaca les prioritats del procés, l’encaix de les recomanacions i les seves limitacions. En el primer cas, els perfils es deriven de variables heterogènies de manera implícita per tal de caracteritzar als usuaris des de múltiples perspectives i punts de vista multidimensionals sense la influència explícita de l’usuari. Això s’aplica al problema d'assignació d’avaluadors per a articles de conferències. Es presta especial atenció al fet de distribuir els avaluadors entre articles per tal de reduir la sobrecàrrega potencial d'una persona i el neguit o el rebuig a la tasca. En el segon cas, les àrees d’interès per a caracteritzar les persones es dedueixen dels seus currículums i s’expressen en termes d’incertesa evitant que els interessos es demanin explícitament a les persones. El sistema s'aplica a un problema de selecció de personal on es posa èmfasi en les preferències del candidat que condueixen a un procés d’encaix asimètric. En el tercer cas, els perfils dels usuaris es defineixen integrant informació implícita i atributs indicats explícitament. Es desenvolupa un model per classificar els ciutadans segons els seus estils de vida que manté la informació original del conjunt de dades del clúster al que ell pertany. Finalment, s’analitzen aquests casos com a proves pilot per generalitzar implementacions en futurs casos reals. Es discuteixen les àrees d'aplicació futures i noves perspectives.Postprint (published version

Knowledge aggregation in people recommender systems : matching skills to tasks

People recommender systems (PRS) are a special type of RS. They are often adopted to identify people capable of performing a task. Recommending people poses several challenges not exhibited in traditional RS. Elements such as availability, overload, unresponsiveness, and bad recommendations can have adverse effects. This thesis explores how people’s preferences can be elicited for single-event matchmaking under uncertainty and how to align them with appropriate tasks. Different methodologies are introduced to profile people, each based on the nature of the information from which it was obtained. These methodologies are developed into three use cases to illustrate the challenges of PRS and the steps taken to address them. Each one emphasizes the priorities of the matching process and the constraints under which these recommendations are made. First, multi-criteria profiles are derived completely from heterogeneous sources in an implicit manner characterizing users from multiple perspectives and multi-dimensional points-of-view without influence from the user. The profiles are introduced to the conference reviewer assignment problem. Attention is given to distribute people across items in order reduce potential overloading of a person, and neglect or rejection of a task. Second, people’s areas of interest are inferred from their resumes and expressed in terms of their uncertainty avoiding explicit elicitation from an individual or outsider. The profile is applied to a personnel selection problem where emphasis is placed on the preferences of the candidate leading to an asymmetric matching process. Third, profiles are created by integrating implicit information and explicitly stated attributes. A model is developed to classify citizens according to their lifestyles which maintains the original information in the data set throughout the cluster formation. These use cases serve as pilot tests for generalization to real-life implementations. Areas for future application are discussed from new perspectives.Els sistemes de recomanació de persones (PRS) són un tipus especial de sistemes recomanadors (RS). Sovint s’utilitzen per identificar persones per a realitzar una tasca. La recomanació de persones comporta diversos reptes no exposats en la RS tradicional. Elements com la disponibilitat, la sobrecàrrega, la falta de resposta i les recomanacions incorrectes poden tenir efectes adversos. En aquesta tesi s'explora com es poden obtenir les preferències dels usuaris per a la definició d'assignacions sota incertesa i com aquestes assignacions es poden alinear amb tasques definides. S'introdueixen diferents metodologies per definir el perfil d’usuaris, cadascun en funció de la naturalesa de la informació necessària. Aquestes metodologies es desenvolupen i s’apliquen en tres casos d’ús per il·lustrar els reptes dels PRS i els passos realitzats per abordar-los. Cadascun destaca les prioritats del procés, l’encaix de les recomanacions i les seves limitacions. En el primer cas, els perfils es deriven de variables heterogènies de manera implícita per tal de caracteritzar als usuaris des de múltiples perspectives i punts de vista multidimensionals sense la influència explícita de l’usuari. Això s’aplica al problema d'assignació d’avaluadors per a articles de conferències. Es presta especial atenció al fet de distribuir els avaluadors entre articles per tal de reduir la sobrecàrrega potencial d'una persona i el neguit o el rebuig a la tasca. En el segon cas, les àrees d’interès per a caracteritzar les persones es dedueixen dels seus currículums i s’expressen en termes d’incertesa evitant que els interessos es demanin explícitament a les persones. El sistema s'aplica a un problema de selecció de personal on es posa èmfasi en les preferències del candidat que condueixen a un procés d’encaix asimètric. En el tercer cas, els perfils dels usuaris es defineixen integrant informació implícita i atributs indicats explícitament. Es desenvolupa un model per classificar els ciutadans segons els seus estils de vida que manté la informació original del conjunt de dades del clúster al que ell pertany. Finalment, s’analitzen aquests casos com a proves pilot per generalitzar implementacions en futurs casos reals. Es discuteixen les àrees d'aplicació futures i noves perspectives

Archives, Access and Artificial Intelligence

Digital archives are transforming the Humanities and the Sciences. Digitized collections of newspapers and books have pushed scholars to develop new, data-rich methods. Born-digital archives are now better preserved and managed thanks to the development of open-access and commercial software. Digital Humanities have moved from the fringe to the center of academia. Yet, the path from the appraisal of records to their analysis is far from smooth. This book explores crossovers between various disciplines to improve the discoverability, accessibility, and use of born-digital archives and other cultural assets

Archives, Access and Artificial Intelligence: Working with Born-Digital and Digitized Archival Collections

Digital archives are transforming the Humanities and the Sciences. Digitized collections of newspapers and books have pushed scholars to develop new, data-rich methods. Born-digital archives are now better preserved and managed thanks to the development of open-access and commercial software. Digital Humanities have moved from the fringe to the center of academia. Yet, the path from the appraisal of records to their analysis is far from smooth. This book explores crossovers between various disciplines to improve the discoverability, accessibility, and use of born-digital archives and other cultural assets

From Anecdotal Evidence to Quantitative Evaluation Methods:A Systematic Review on Evaluating Explainable AI

The rising popularity of explainable artificial intelligence (XAI) to understand high-performing black boxes, also raised the question of how to evaluate explanations of machine learning (ML) models. While interpretability and explainability are often presented as a subjectively validated binary property, we consider it a multi-faceted concept. We identify 12 conceptual properties, such as Compactness and Correctness, that should be evaluated for comprehensively assessing the quality of an explanation. Our so-called Co-12 properties serve as categorization scheme for systematically reviewing the evaluation practice of more than 300 papers published in the last 7 years at major AI and ML conferences that introduce an XAI method. We find that 1 in 3 papers evaluate exclusively with anecdotal evidence, and 1 in 5 papers evaluate with users. We also contribute to the call for objective, quantifiable evaluation methods by presenting an extensive overview of quantitative XAI evaluation methods. This systematic collection of evaluation methods provides researchers and practitioners with concrete tools to thoroughly validate, benchmark and compare new and existing XAI methods. This also opens up opportunities to include quantitative metrics as optimization criteria during model training in order to optimize for accuracy and interpretability simultaneously.Comment: Link to website added: https://utwente-dmb.github.io/xai-papers

Archives, Access and Artificial Intelligence

Digital archives are transforming the Humanities and the Sciences. Digitized collections of newspapers and books have pushed scholars to develop new, data-rich methods. Born-digital archives are now better preserved and managed thanks to the development of open-access and commercial software. Digital Humanities have moved from the fringe to the center of academia. Yet, the path from the appraisal of records to their analysis is far from smooth. This book explores crossovers between various disciplines to improve the discoverability, accessibility, and use of born-digital archives and other cultural assets

Survey over Existing Query and Transformation Languages

A widely acknowledged obstacle for realizing the vision of the Semantic Web is the inability of many current Semantic Web approaches to cope with data available in such diverging representation formalisms as XML, RDF, or Topic Maps. A common query language is the first step to allow transparent access to data in any of these formats. To further the understanding of the requirements and approaches proposed for query languages in the conventional as well as the Semantic Web, this report surveys a large number of query languages for accessing XML, RDF, or Topic Maps. This is the first systematic survey to consider query languages from all these areas. From the detailed survey of these query languages, a common classification scheme is derived that is useful for understanding and differentiating languages within and among all three areas

Synthesizing Qualitative Evidence: A Roadmap for Information Systems Research

Qualitative synthesis research is an approach that consolidates the output of different qualitative studies to create new subject knowledge. Such work can help reveal more powerful explanations than that seen in a single study, thereby generating increased levels of understanding of a given phenomenon and greater research finding generalizability. Based on a review of the literature and a survey of qualitative researchers, we found that the information systems (IS) domain lacks a clear understanding of qualitative synthesis methods and, as a result, has largely failed to take advantage of this powerful, high-potential methodological opportunity. To address this shortcoming, this paper is the first to provide a rigorous overview of the full suite of 35 qualitative synthesis methods, as well as guidelines that include a three-tiered selection framework. By using the guidelines and framework in tandem, IS researchers are able to select the qualitative synthesis method most appropriate for a given research study, particularly when the research objective involves knowledge integration/aggregation, interpretation/theory development, and/or informing IS practice

A systematic review of local vulnerability to climate change: in search of transparency, coherence and comparability

Because vulnerability is a conceptual construct rather than a directly observable phenomenon, most vulnerability assessments measure a set of “vulnerability indicators”. In order to identify the core approaches and range of variation in the field, we conducted a systematic literature review on local vulnerability to climate change. The systematic review entailed an identification of frameworks, concepts, and operationalizations and a transparency assessment of their reporting. Three fully defined relevant frameworks of vulnerability were identified: IPCC, Patterns of Smallholder Vulnerability and Vulnerability as Expected Poverty. Comparative analysis found substantial heterogeneity in frameworks, concepts and operationalizations, making it impossible to identify patterns of climate vulnerability indicators and determinants that have robust empirical support. If research measuring farmers’ vulnerability to climate change is to have any comparability, it needs greater conceptual coherence and empirical validity. We recommend a systematic program of testing and validating vulnerability measures before institutionalizing them in programmatic contexts