A Bisociated Domain-Based Serendipitous Novelty-Recommendation Technique for Recommender Systems

Traditional recommendation paradigms such as content-based filtering (CBF) tend to recommend items that are very similar to user profile characteristics and item input, resulting in the classical twin problem of overspecialization and concentration bias of recommendations. This twin problem is prevalent with CBF recommender systems due to the utilisation of accuracy metrics to retrieve similar items, and, limiting recommendation computations to single recognized user-centered domains, rather than cross-domains.  This paper proposes a Bisociated domain-based serendipitous novelty recommendation techniques using Bisolinkers exploratory creativity discovery technique. The use of Bisolinkers enables establishing unique links between two seemingly unrelated domains, to enhance recommendation accuracy and user satisfaction. The presence of similar terms in two habitually incompatible domains demonstrates that two seemingly unrelated domains contain elements that are related and may act as a link to connect these two domains. Keywords: recommender systems, novelty, machine learning, outlier detection, bisociation &nbsp

An Assessment of Impact Metrics’ Potential as Research Indicators Based on Their Perception, Usage, and Dependencies from External Science Communication

The demand for practicable methods for quantitative assessments of scientific products’ relevance has risen considerably over the past decades. As a consequence, research and commercial providers of scholarly data developed a wide variety of impact indicators, ranging from citation-based to so-called altmetrics. This highly heterogeneous family of indicators is based on the principle of measuring interactions with scientific publications that are observable online, and covers for instance mentions of publications in social and journalistic media, in literature management software, or in policy documents. The various metrics' theoretical validity as impact indicators is debated constantly, as questions regarding what it is that different metrics measure or express in many facets remain unanswered. This thesis makes two central contributions towards answering these questions. Its first part systematically assesses the status quo of various metrics’ perception and usage by researchers. This assessment serves to determine the significance of metrics in academic daily routines, as well as to identify relevant perceived problems concerning their usage. The challenges identified this way are in later sections of the thesis opposed with concrete measures to be taken during the development of future research metrics and their infrastructure to effectively solve common criticisms regarding current metrics and their use. Proceeding from the first part’s user studies, this thesis’ second part examines the relationship between research metrics and external science communication. It this way addresses a wide research gap with considerable potential implications for metrics’ validity as indicators for quality - the question to which degree these metrics are merely the result of promotion, which respective research publications receive

Biases in scholarly recommender systems: impact, prevalence, and mitigation

We create a simulated financial market and examine the effect of different levels of active and passive investment on fundamental market efficiency. In our simulated market, active, passive, and random investors interact with each other through issuing orders. Active and passive investors select their portfolio weights by optimizing Markowitz-based utility functions. We find that higher fractions of active investment within a market lead to an increased fundamental market efficiency. The marginal increase in fundamental market efficiency per additional active investor is lower in markets with higher levels of active investment. Furthermore, we find that a large fraction of passive investors within a market may facilitate technical price bubbles, resulting in market failure. By examining the effect of specific parameters on market outcomes, we find that that lower transaction costs, lower individual forecasting errors of active investors, and less restrictive portfolio constraints tend to increase fundamental market efficiency in the market

Mapping Scholarly Communication Infrastructure: A Bibliographic Scan of Digital Scholarly Communication Infrastructure

This bibliography scan covers a lot of ground. In it, I have attempted to capture relevant recent literature across the whole of the digital scholarly communications infrastructure. I have used that literature to identify significant projects and then document them with descriptions and basic information. Structurally, this review has three parts. In the first, I begin with a diagram showing the way the projects reviewed fit into the research workflow; then I cover a number of topics and functional areas related to digital scholarly communication. I make no attempt to be comprehensive, especially regarding the technical literature; rather, I have tried to identify major articles and reports, particularly those addressing the library community. The second part of this review is a list of projects or programs arranged by broad functional categories. The third part lists individual projects and the organizations—both commercial and nonprofit—that support them. I have identified 206 projects. Of these, 139 are nonprofit and 67 are commercial. There are 17 organizations that support multiple projects, and six of these—Artefactual Systems, Atypon/Wiley, Clarivate Analytics, Digital Science, Elsevier, and MDPI—are commercial. The remaining 11—Center for Open Science, Collaborative Knowledge Foundation (Coko), LYRASIS/DuraSpace, Educopia Institute, Internet Archive, JISC, OCLC, OpenAIRE, Open Access Button, Our Research (formerly Impactstory), and the Public Knowledge Project—are nonprofit.Andrew W. Mellon Foundatio

Illusions of a ‘Bond’: Tagging Cultural Products across Online Platforms

Structured Abstract Purpose Most studies pertaining to social tagging focus on one platform or platform type, thus limiting the scope of their findings. This study explores social tagging practices across four platforms in relation to cultural products associated with the book Casino Royale, by Ian Fleming. Design/methodology/approach A layered and nested case study approach was used to analyze data from four online platforms: Goodreads, Last.fm, WordPress, and public library social discovery platforms. The top-level case study focuses on the book Casino Royale, by Ian Fleming, and its derivative products. The analysis of tagging practices in each of the four online platforms is nested within the top-level case study. ‘Casino Royale’ was conceptualized as a cultural product (the book), its derived products (e.g., movies, theme songs), as well as a keyword in blogs. A qualitative, inductive, and context-specific approach was chosen to identify commonalities in tagging practices across platforms whilst taking into account the uniqueness of each platform. Findings The four platforms comprise different communities of users, each platform with its own cultural norms and tagging practices. Traditional access points in the library catalogues focused on the subject, location, and fictitious characters of the book. User-generated content across the four platforms emphasized historical events and periods related to the book, and highlighted more subjective access points, such as recommendations, tone, mood, reaction, and reading experience. Revealing shifts occur in the tags between the original book and its cultural derivatives: Goodreads and library catalogues focus almost exclusively on the book, while Last.fm and WordPress make additional cross-references to a wider range of different cultural products, including books, movies, and music. The analyses also yield apparent similarities in certain platforms, such as recurring terms, phrasing and composite or multifaceted tags, as well as a strong presence of genre-related terms for the book and music. Originality/value The layered and nested case study approach presents a more comprehensive theoretical viewpoint and methodological framework by which to explore the study of user-generated metadata pertaining to a range of related cultural products across a variety of online platforms

Recommendations in Academic Social Media: the shaping of scholarly communication through algorithmic mediation

Scholarly communication is increasingly being mediated by Academic Social Media (ASM) platforms, which combine the functions of a scientifi c repository with social media features such as personal profi les, followers and comments. In ASM, algorithmic mediation is responsible for fi ltering the content and distributing it in personalised individual feeds and recommendations according to inferred relevance to users. However, if communication among researchers is intertwined with these platforms, in what ways may the recommendation algorithms in ASM shape scholarly communication? Scientifi c literature has been investigating how content is mediated in data-driven environments ranging from social media platforms to specifi c apps, whereas algorithmic mediation in scientifi c environments remains neglected. This thesis starts from the premise that ASM platforms are sociocultural artefacts embedded in a mutually shaping relationship with research practices and economic, political and social arrangements. Therefore, implications of algorithmic mediation can be studied through the artefact itself, peoples’ practices and the social/political/ economic arrangements that aff ect and are aff ected by such interactions. Most studies on ASM focus on one of these elements at a time, either examining design elements or the users’ behaviour on and perceptions about such platforms. In this thesis, a multifaceted approach is taken to analyse the artefact as well as the practices and arrangements traversed by algorithmic mediation. Chapter 1 reviews the literature about ASM platforms, and explains the history of algorithmic recommendations, starting from the fi rst Information Retrieval systems to current Recommender Systems, highlighting the use of diff erent data sources and techniques. The chapter also presents the mediation framework and how it applies to ASM platforms, before outlining the thesis. The rest of the thesis is divided in two parts. Part I focuses on how recommender systems in ASM shape what users can see and how users interact with and through the platform. Part II investigates how, in turn, researchers make sense of their online interactions within ASM. The end of Chapter 1 shows the methodological choices for each following chapter. Part I presents a case study of one of the most popular ASM platforms in which a walkthrough method was conducted in four steps (interface analysis, web code inspection, patent analysis and company inquiry using the General Data Protection Regulation (GDPR)). In Chapter 2 it is shown that almost all the content in ASM platforms are algorithmically mediated through mechanisms of profi ling, information selection and commodifi cation. It is also discussed how the company avoids explaining the workings of recommender systems and the mutually shaping characteristic of ASM platforms. Chapter 3 explores the distortions and biases that ASM platforms can uphold. Results show how profi ling, datafi cation and prioritization have the potential to foster homogeneity bias, discrimination, the Matthew eff ect of cumulative advantage in science and other distortions. Part II consists of two empirical studies involving participants from diff erent countries in interviews (n=11) and a research game (n=13). Chapter 4 presents the interviews combined with the show and tell technique. The results show the participant’s perceptions on ASM aff ordances, that revolve around six main themes: (1) getting access to relevant content; (2) reaching out to other scholars; (3) algorithmic impact on exposure to content; (4) to see and to be seen; (5) blurred boundaries of potential ethical or legal infringements, and (6) the more I give, the more I get. We argue that algorithmic mediation not only constructs a narration of the self, but also a narration of the relevant other in ASM platforms, confi guring an image of the relevant other that is both participatory and productive. Chapter 5 presents the design process of a research game and the results of the empirical sessions, where participants were observed while playing the game. There are two outcomes for the study. First, the human values researchers relate to algorithmic features in ASM, the most prominent being stimulation, universalism and self-direction. Second, the role of the researcher’s approach (collaborative, competitive or ambivalent) in academic tasks, showing the consequential choices people make regarding algo- rithmic features and the motivations behind those choices. The results led to four archetypal profi les: (1) the collaborative reader; (2) the competitive writer; (3) the collaborative disseminator; and (4) the ambivalent evaluator. The fi nal chapter summarises the ways in which ASM platforms forges people’s perceptions and the strategies people employ to use the systems in benefi t of their careers, answering each research question. Chapter 6 discusses the implications of algorithmic mediation for scholarly communication and science in general. The dissertation ends with refl ections on human agency in data-driven environments, the role of algorithmic inferences in science and the challenge of reconciling individual user’s needs with broader goals of the scientifi c community. By doing so, the contribution of this thesis is twofold, (1) providing in-depth knowledge about the ASM artefact, and (2) unfolding diff erent aspects of the human perspective in dealing with algorithmic mediation in ASM. Both perspectives are discussed in light of social arrangements that are mutually shaped by artefact and practices.A comunicação acadêmica é cada vez mais mediada por plataformas de Mídia Social Acadêmica (MSA), que combinam as funções de um repositório científi co com recursos de mídia social, como perfi s pessoais, seguidores e comentários. Nas MSA, a mediação algorítmica é responsável por fi ltrar o conteúdo e distribuí-lo em feeds e recomendações individuais personalizados de acordo com a relevância inferida para os usuários. No entanto, se a comunicação entre pesquisadores está entrelaçada com essas plataformas, de que forma os algoritmos de recomendação nas MSA podem moldar a comunicação acadêmica? A literatura científi ca vem investigando como o conteúdo é mediado em ambientes orientados por dados, desde plataformas de mídia social até aplicativos específi cos, enquanto a mediação algorítmica em ambientes científi cos permanece negligenciada. Esta tese parte da premissa de que as plataformas de MSA são artefatos socioculturais inseridos em uma relação mutuamente modeladora com práticas de pesquisa e arranjos econômicos, políticos e sociais. Portanto, as implicações da mediação algorítmica podem ser estudadas através do próprio artefato, das práticas humanas e dos arranjos sociais/políticos/ econômicos que afetam e são afetados por tais interações. A maioria dos estudos sobre MSA se concentra em um desses elementos de cada vez, seja examinando elementos de design ou o comportamento e percepções dos usuários sobre essas plataformas. Nesta tese, uma abordagem multifacetada é feita para analisar o artefato, bem como as práticas e arranjos atravessados pela mediação algorítmica. O Capítulo 1 revisa a literatura sobre plataformas de MSA e explica a história das recomendações algorítmicas, desde os primeiros sistemas de Recuperação de Informação até os atuais Sistemas de Recomendação, destacando o uso de diferentes fontes de dados e técnicas. O capítulo também apresenta o quadro teórico (mediation framework) e como ele se aplica às plataformas MSA, antes de delinear a estrutura da tese. O restante da tese está dividido em duas partes. A Parte I se concentra em como os sistemas de recomendação nas MSA moldam o que os usuários podem ver e como os usuários interagem com e na plataforma. A Parte II, por sua vez, investiga como os pesquisadores dão sentido às suas interações online dentro das MSA. O fi nal do Capítulo 1 mostra as opções metodológicas para cada capítulo seguinte. A Parte I apresenta um estudo de caso de uma das plataformas de MSA mais populares em que o walkthrough method foi realizado em quatro etapas (análise de interface, inspeção de código web, análise de patente e consulta à empresa usando o General Data Protection Regulation (GDPR)). No Capítulo 2 é mostrado que quase todo o conteúdo das plataformas ASM é mediado por algoritmos por meio de mecanismos de perfi - lamento, seleção de informações e mercantilização. Também é discutido como a empresa evita explicar o funcionamento dos sistemas de recomendação e a característica de modelagem mútua das plataformas de MSA. O Capítulo 3 explora as distorções e vieses que as plataformas de MSA podem sustentar. Os resultados mostram como o perfi lamento, a datifi cação e a priorização de conteúdo têm o potencial de promover viés de homogeneidade, discriminação o efeito Mateus de vantagem cumulativa na ciência e outras distorções. A Parte II consiste em dois estudos empíricos envolvendo participantes de diferentes países em entrevistas (n=11) e um jogo de pesquisa (n=13). O capítulo 4 apresenta as entrevistas combinadas com a técnica show and tell. Os resultados mostram as percepções dos participantes sobre as aff ordances das MSA, que giram em torno de seis temas principais: (1) ter acesso a conteúdos relevantes; (2) acesso a outros pesquisadores; (3) impacto algorítmico na exposição ao conteúdo; (4) ver e ser visto; (5) limites difusos de potenciais infrações éticas ou legais e (6) quanto mais eu dou, mais eu recebo. Argumentamos que a mediação algorítmica não apenas constrói uma narração do eu, mas também uma narração do outro nas plataformas de MSA, confi gurando uma imagem do outro ao mesmo tempo participativa e produtiva. O capítulo 5 apresenta o processo de design de um jogo de pesquisa e os resultados das sessões empíricas, onde os participantes foram observados enquanto jogavam o jogo. Há dois resultados para o estudo. Primeiro, quais valores humanos os pesquisadores relacionam com recursos algorítmicos nas MSA, sendo os mais proeminentes o estímulo, o universalismo e o autodirecionamento. Em segundo lugar, o papel da abordagem do pesquisador (colaborativa, competitiva ou ambivalente) em tarefas acadêmicas, mostrando as escolhas consequentes que as pessoas fazem em relação aos recursos algorítmicos e as motivações por trás dessas escolhas. Os resultados levaram a quatro perfi s arquetípicos: (1) o leitor colaborativo; (2) o escritor competitivo; (3) o divulgador colaborativo; e (4) o avaliador ambivalente. O capítulo fi nal (Capítulo 6) resume as maneiras pelas quais as plataformas de MSA forjam as percepções das pessoas e as estratégias que as pessoas empregam para usar os sistemas em benefício de suas carreiras, respondendo a cada questão de pesquisa. O capítulo discute ainda as implicações da mediação algorítmica para a comunicação acadêmica e a ciência em geral. A dissertação termina com refl exões sobre a agência humana em ambientes orientados por dados, o papel das inferências algorítmicas na ciência e o desafi o de conciliar as necessidades individuais do usuário com os objetivos mais amplos da comunidade científi ca. Ao fazê-lo, a contribuição desta tese é dupla, (1) fornecendo conhecimento aprofundado sobre o artefato plataformas de MSA, e (2) desdobrando diferentes aspectos da perspectiva humana ao lidar com mediação algorítmica em ASM. Ambas as perspectivas são discutidas à luz de arranjos sociais que são mutuamente moldados por artefatos e práticas

Study on open science: The general state of the play in Open Science principles and practices at European life sciences institutes

Nowadays, open science is a hot topic on all levels and also is one of the priorities of the European Research Area. Components that are commonly associated with open science are open access, open data, open methodology, open source, open peer review, open science policies and citizen science. Open science may a great potential to connect and influence the practices of researchers, funding institutions and the public. In this paper, we evaluate the level of openness based on public surveys at four European life sciences institute

B!SON: A Tool for Open Access Journal Recommendation

Finding a suitable open access journal to publish scientific work is a complex task: Researchers have to navigate a constantly growing number of journals, institutional agreements with publishers, funders’ conditions and the risk of Predatory Publishers. To help with these challenges, we introduce a web-based journal recommendation system called B!SON. It is developed based on a systematic requirements analysis, built on open data, gives publisher-independent recommendations and works across domains. It suggests open access journals based on title, abstract and references provided by the user. The recommendation quality has been evaluated using a large test set of 10,000 articles. Development by two German scientific libraries ensures the longevity of the project

Estimating credibility of science claims : analysis of forecasting data from metascience projects : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Statistics at Massey University, Albany, New Zealand

The veracity of scientific claims is not always certain. In fact, sufficient claims have been proven incorrect that many scientists believe that science itself is facing a “replication crisis”. Large scale replication projects provided empirical evidence that only around 50% of published social and behavioral science findings are replicable. Multiple forecasting studies showed that the outcomes of replication projects could be predicted by crowdsourced human evaluators. The research presented in this thesis builds on previous forecasting studies, deriving new findings and exploring new scope and scale. The research is centered around the DARPA SCORE (Systematizing Confidence in Open Research and Evidence) programme, a project aimed at developing measures of credibility for social and behavioral science claims. As part of my contribution to SCORE, myself, along with a international collaboration, elicited forecasts from human experts via surveys and prediction markets to predict the replicability of 3000 claims. I also present research on other forecasting studies. In chapter 2, I pool data from previous studies to analyse the performance of prediction markets and surveys with higher statistical power. I confirm that prediction markets are better at forecasting replication outcomes than surveys. This study also demonstrates the relationship between p-values of original findings and replication outcomes. These findings are used to inform the experimental and statistical design to forecast the replicability of 3000 claims as part of the SCORE programme. A full description of the design including planned statistical analyses is included in chapter 3. Due to COVID-19 restrictions, our generated forecasts could not be validated through direct replication, experiments conducted by other teams within the SCORE collaboration, thereby preventing results being presented in this thesis. The completion of these replications is now scheduled for 2022, and the pre-analysis plan presented in Chapter 3 will provide the basis for the analysis of the resulting data. In chapter 4, an analysis of ‘meta’ forecasts, or forecasts regarding field wide replication rates and year specific replication rates, is presented. We presented and published community expectations that replication rates will differ by field and will increase over time. These forecasts serve as valuable insights into the academic community’s views of the replication crisis, including those research fields for which no large-scale replication studies have been undertaken yet. Once the full results from SCORE are available, there will be additional insights from validations of the community expectations. I also analyse forecaster’s ability to predict replications and effect sizes in Chapters 5 (Creative Destruction in Science) and 6 (A creative destruction approach to replication: Implicit work and sex morality across cultures). In these projects a ‘creative destruction’ approach to replication was used, where a claim is compared not only to the null hypothesis but to alternative contradictory claims. I conclude forecasters can predict the size and direction of effects. Chapter 7 examines the use of forecasting for scientific outcomes beyond replication. In the COVID-19 preprint forecasting project I find that forecasters can predict if a preprint will be published within one year, including the quality of the publishing journal. Forecasters can also predict the number of citations preprints will receive. This thesis demonstrates that information about scientific claims with respect to replicability is dispersed within scientific community. I have helped to develop methodologies and tools to efficiently elicit and aggregate forecasts. Forecasts about scientific outcomes can be used as guides to credibility, to gauge community expectations and to efficiently allocate sparse replication resources

A guide for social science journal editors on easing into open science

Journal editors have a large amount of power to advance open science in their respective fields by incentivising and mandating open policies and practices at their journals. The Data PASS Journal Editors Discussion Interface (JEDI, an online community for social science journal editors: www.dpjedi.org) has collated several resources on embedding open science in journal editing (www.dpjedi.org/resources). However, it can be overwhelming as an editor new to open science practices to know where to start. For this reason, we created a guide for journal editors on how to get started with open science. The guide outlines steps that editors can take to implement open policies and practices within their journal, and goes through the what, why, how, and worries of each policy and practice. This manuscript introduces and summarizes the guide (full guide: https://osf.io/hstcx)