FindYourHelp: an expert finder module on Virtual Learning Environments

Este artigo discute a criação de um módulo adicional para o ambiente Moodle, que permita a identificação automática de especialistas que contribuam em seus fóruns de discussão. O diferencial da solução desenvolvida consiste em aplicar técnicas de mineração de textos como forma complementar à análise de participação dos estudantes já existente no ambiente. Um estudo de viabilidade da solução foi desenvolvido e evidenciou a aplicabilidade da ferramenta em relação a seus objetivos iniciais

Expertise Profiling in Evolving Knowledgecuration Platforms

Expertise modeling has been the subject of extensiveresearch in two main disciplines: Information Retrieval (IR) andSocial Network Analysis (SNA). Both IR and SNA approachesbuild the expertise model through a document-centric approachproviding a macro-perspective on the knowledge emerging fromlarge corpus of static documents. With the emergence of the Webof Data there has been a significant shift from static to evolvingdocuments, through micro-contributions. Thus, the existingmacro-perspective is no longer sufficient to track the evolution ofboth knowledge and expertise. In this paper we present acomprehensive, domain-agnostic model for expertise profiling inthe context of dynamic, living documents and evolving knowledgebases. We showcase its application in the biomedical domain andanalyze its performance using two manually created datasets

Through the Looking-Glass: Transparency Implications and Challenges in Enterprise AI Knowledge Systems

Knowledge can't be disentangled from people. As AI knowledge systems mine vast volumes of work-related data, the knowledge that's being extracted and surfaced is intrinsically linked to the people who create and use it. When these systems get embedded in organizational settings, the information that is brought to the foreground and the information that's pushed to the periphery can influence how individuals see each other and how they see themselves at work. In this paper, we present the looking-glass metaphor and use it to conceptualize AI knowledge systems as systems that reflect and distort, expanding our view on transparency requirements, implications and challenges. We formulate transparency as a key mediator in shaping different ways of seeing, including seeing into the system, which unveils its capabilities, limitations and behavior, and seeing through the system, which shapes workers' perceptions of their own contributions and others within the organization. Recognizing the sociotechnical nature of these systems, we identify three transparency dimensions necessary to realize the value of AI knowledge systems, namely system transparency, procedural transparency and transparency of outcomes. We discuss key challenges hindering the implementation of these forms of transparency, bringing to light the wider sociotechnical gap and highlighting directions for future Computer-supported Cooperative Work (CSCW) research

Multivariate Fairness for Paper Selection

Peer review is the process by which publishers select the best publications for inclusion in a journal or a conference. Bias in the peer review process can impact which papers are selected for inclusion in conferences and journals. Although often implicit, race, gender and other demographics can prevent members of underrepresented groups from presenting at major conferences. To try to avoid bias, many conferences use a double-blind review process to increase fairness during reviewing. However, recent studies argue that the bias has not been removed completely. Our research focuses on developing fair algorithms that correct for these biases and select papers from a more demographically diverse group of authors. To address this, we present fair algorithms that explicitly incorporate author diversity in paper recommendation using multidimensional author profiles that include five demographic features, i.e., gender, ethnicity, career stage, university rank, and geolocation. The Overall Diversity method ranks papers based on an overall diversity score whereas the Multifaceted Diversity method selects papers that fill the highest-priority demographic feature first. We evaluate these algorithms with Boolean and continuous-valued features by recommending papers for SIGCHI 2017 from a pool of SIGCHI 2017, DIS 2017 and IUI 2017 papers and compare the resulting set of papers with the papers accepted by the conference. Both methods increase diversity with small decreases in utility using profiles with either Boolean or continuous feature values. Our best method, Multifaceted Diversity, recommends a set of papers that match demographic parity, selecting authors who are 42.50% more diverse with a 2.45% gain in utility. This approach could be applied when selecting conference papers, journal papers, grant proposals, or other tasks within academia

Multivariate Fairness for Paper Selection

Peer review is the process by which publishers select the best publications for inclusion in a journal or a conference. Bias in the peer review process can impact which papers are selected for inclusion in conferences and journals. Although often implicit, race, gender and other demographics can prevent members of underrepresented groups from presenting at major conferences. To try to avoid bias, many conferences use a double-blind review process to increase fairness during reviewing. However, recent studies argue that the bias has not been removed completely. Our research focuses on developing fair algorithms that correct for these biases and select papers from a more demographically diverse group of authors. To address this, we present fair algorithms that explicitly incorporate author diversity in paper recommendation using multidimensional author profiles that include five demographic features, i.e., gender, ethnicity, career stage, university rank, and geolocation. The Overall Diversity method ranks papers based on an overall diversity score whereas the Multifaceted Diversity method selects papers that fill the highest-priority demographic feature first. We evaluate these algorithms with Boolean and continuous-valued features by recommending papers for SIGCHI 2017 from a pool of SIGCHI 2017, DIS 2017 and IUI 2017 papers and compare the resulting set of papers with the papers accepted by the conference. Both methods increase diversity with small decreases in utility using profiles with either Boolean or continuous feature values. Our best method, Multifaceted Diversity, recommends a set of papers that match demographic parity, selecting authors who are 42.50% more diverse with a 2.45% gain in utility. This approach could be applied when selecting conference papers, journal papers, grant proposals, or other tasks within academia

Fair and Diverse Group Formation Based on Multidimensional Features

The goal of group formation is to build a team to accomplish a specific task. Algorithms are being developed to improve the team\u27s effectiveness so formed and the efficiency of the group selection process. However, there is concern that team formation algorithms could be biased against minorities due to the algorithms themselves or the data on which they are trained. Hence, it is essential to build fair team formation systems that incorporate demographic information into the process of building the group. Although there has been extensive work on modeling individuals’ expertise for expert recommendation and/or team formation, there has been relatively little prior work on modeling demographics and incorporating demographics into the group formation process. We propose a novel method to represent experts’ demographic profiles based on multidimensional demographic features. Moreover, we introduce three diversity ranking algorithms that form a group by considering demographic features along with the minimum required skills. Unlike many ranking algorithms that consider one Boolean demographic feature (e.g., gender or race), our diversity ranking algorithms consider multiple demographic features simultaneously. Finally, we introduce a fair team formation algorithm that balances each candidate\u27s demographic information and expertise. We evaluate our proposed algorithms using real datasets based on members of a computer science program committee. The result shows that our algorithms form a program committee that is more diverse with an acceptable loss in utility

Integrating multiple windows and document features for expert finding

Expert finding is a key task in enterprise search and has recently attracted lots of attention from both research and industry communities. Given a search topic, a prominent existing approach is to apply some information retrieval (IR) system to retrieve top ranking documents, which will then be used to derive associations between experts and the search topic based on cooccurrences. However, we argue that expert finding is more sensitive to multiple levels of associations and document features that current expert finding systems insufficiently address, including (a) multiple levels of associations between experts and search topics, (b) document internal structure, and (c) document authority. We propose a novel approach that integrates the above-mentioned three aspects as well as a query expansion technique in a two-stage model for expert finding. A systematic evaluation is conducted on TREC collections to test the performance of our approach as well as the effects of multiple windows, document features, and query expansion. These experimental results show that query expansion can dramatically improve expert finding performance with statistical significance. For three well-known IR models with or without query expansion, document internal structures help improve a single window-based approach but without statistical significance, while our novel multiple window-based approach can significantly improve the performance of a single window-based approach both with and without document internal structures

Um modelo para extração de perfil de especialista aplicado às ferramentas de expertise location e apoio à Gestão do Conhecimento

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós-Graduação em Engenharia e Gestão do Conhecimento, Florianópolis, 2016.As ferramentas de Expertise Location podem ser utilizadas amplamente na Gestão do Conhecimento para apoiar a identificação e o compartilhamento do conhecimento. Porém, manter os dados dos colaboradores de uma organização atualizados nessas ferramentas pode ser desafiador. Muitas vezes, os colaboradores precisam preencher as mesmas informações em diversos sistemas. Como uma abordagem alternativa para simplificar esse processo de atualização dos dados, este trabalho propõe um modelo para a extração automática de perfis de especialistas a partir de seus documentos não estruturados. Assim, realizou-se uma pesquisa aplicada e exploratória com base em uma revisão integrativa da literatura, a qual resultou na identificação das abordagens atuais para a extração de perfil de especialista que permitisse a construção desse modelo. A partir dessas abordagens, foram elaborados um modelo conceitual e um protótipo baseados em Processamento de Linguagem Natural para a tarefa de extração de informações de perfil de especialistas que possam fornecer insumos para a identificação de seus conhecimentos e de suas áreas de interesse. A implementação do protótipo resultou também em uma ferramenta de código aberto. Tal ferramenta é disponibilizada em um site público, em conjunto com o seu código-fonte, e gera uma página de perfil com o uso de componentes de tag cloud e timeline. Com o intuito de verificar a viabilidade do modelo proposto, a partir de documentos de voluntários, foram executados testes comparando os perfis gerados pela ferramenta com os perfis presentes na rede social LinkedIn. Os resultados dos testes demonstraram que o modelo proposto pode representar uma alternativa viável para a geração de perfis de especialistas de forma automática com o objetivo de apoiar as ferramentas de Expertise Location em uma organização. Consequentemente, a adoção desse modelo pode reduzir a necessidade de atualizações constantes dos perfis de especialistas de forma manual.Abstract : The Expertise Location Tools can be widely used in Knowledge Management in order to support the identification and sharing of the knowledge. However, to keep the data of the employees of an organization updated in those tools can be challenging. From time to time, employees need to fill out the same data in different systems. As an alternative approach to simplify this process of updating the data, this paper proposes a model for the automatic extraction of expert profiles from their own non-structural documents. Thus, an applied and exploratory research based on an integrative literature review was carried out, resulting in the identification of the current approaches to the extraction of an expert profile that could allow the construction of this model. From these approaches were elaborated a conceptual model and a prototype based on Natural Language Processing for the task of extraction of information from expert profiles that could provide inputs to the identification of their expertise and their areas of interest. The prototype implementation has also resulted in an open source tool. This tool is available on a public website together with its source code and it generates a profile page using the tag cloud and timeline components. In order to verify the feasibility of the proposed model, tests from documents of volunteers were performed comparing the profiles generated by the tool with those profiles on LinkedIn social network. The test results demonstrated that the proposed model can represent a viable alternative to the generation of automatically expert profiles in order to support Expertise Location tools in an organization. Consequently, the adoption of this model can reduce the need for constant updates of the expert profiles

Promoting Diversity in Academic Research Communities Through Multivariate Expert Recommendation

Expert recommendation is the process of identifying individuals who have the appropriate knowledge and skills to achieve a specific task. It has been widely used in the educational environment mainly in the hiring process, paper-reviewer assignment, and assembling conference program committees. In this research, we highlight the problem of diversity and fair representation of underrepresented groups in expertise recommendation, factors that current expertise recommendation systems rarely consider. We introduce a novel way to model experts in academia by considering demographic attributes in addition to skills. We use the h-index score to quantify skills for a researcher and we identify five demographic features with which to represent a researcher\u27s demographic profile. We highlight the importance of these features and their role in bias within the academic environment. We utilize these demographic features within an expert recommender system in academia to achieve demographic diversity and increase the exposure of the underrepresented groups using two approaches. In the first approach, we present three different algorithms for scholar recommendation: expertise-based, diversity-based, and a hybrid algorithm that uses a tuning parameter to calibrate the balance between expertise loss and diversity gain. To evaluate the ranking produced by these algorithms, we introduce a modified normalized Discounted Cumulative Gain (nDCG) version that supports multi-dimensional features, and we report diversity gain from each method. Our results show that we can achieve the best possible balance between diversity gain and expertise loss when the tuning parameter value is set around 0.4, giving nearly equal weight to both expertise and diversity. Finally, we explore diversity from the lens of the demographic parity and develop two algorithms to achieve a representative group that reflects the demographics of the recommendation pool. One is inspired by Hill Climbing, a mathematical optimization technique, wherein a solution is built gradually to the problem, and the other one is inspired by the problem of seat allocation in electoral voting systems. We evaluated these algorithms by comparing them to the hybrid algorithm from the previous approach. Our evaluation shows that both approaches provide a better diversity gain as compared to the hybrid algorithm. However, Hill Climbing Diversity is more effective when it comes to expertise savings with a statistically significant result, making it the preferred algorithm to achieve the goal of promoting diversity while maintaining expertise in an expert recommendation process