Mining Scholarly Publications for Research Evaluation

Scientific research can lead to breakthroughs that revolutionise society by solving long-standing problems. However, investment of public funds into research requires the ability to clearly demonstrate beneficial returns, accountability, and good management. At the same time, with the amount of scholarly literature rapidly expanding, recognising key research that presents the most important contributions to science is becoming increasingly difficult and time-consuming. This creates a need for effective and appropriate research evaluation methods. However, the question of how to evaluate the quality of research outcomes is very difficult to answer and despite decades of research, there is still no standard solution to this problem. Given this growing need for research evaluation, it is increasingly important to understand how research should be evaluated, and whether the existing methods meet this need. However, the current solutions, which are predominantly based on counting the number of interactions in the scholarly communication network, are insufficient for a number of reasons. In particular, they struggle in capturing many aspects of the academic culture and often significantly lag behind current developments. This work focuses on the evaluation of research publications and aims at creating new methods which utilise publication content. It studies the concept of research publication quality, methods assessing the performance of new research publication evaluation methods, analyses and extends the existing methods, and, most importantly, presents a new class of metrics which are based on publication manuscripts. By bridging the fields of research evaluation and text- and data-mining, this work provides tools for analysing the outcomes of research, and for relieving information overload in scholarly publishing

Altmetrics for Digital Libraries: Concepts, Applications, Evaluation, and Recommendations

The volume of scientific literature is rapidly increasing, which has led to researchers becoming overloaded by the number of articles that they have available for reading and difficulties in estimating their quality and relevance (e.g., based on their research interests). Library portals, in these circumstances, are increasingly getting more relevant by using quality indicators that can help researchers during their research discovery process. Several evaluation methods (e.g., citations, Journal Impact Factor, and peer-reviews) have been used and suggested by library portals to help researchers filter out the relevant articles (e.g., articles that have received high citations) for their needs. However, in some cases, these methods have been criticized, and a number of weaknesses have been identified and discussed. For example, citations usually take a long time to appear, and some articles that are important can remain uncited. With the growing presence of social media today, new alternative indicators, known as “altmetrics,” have been encountered and proposed as complementary indicators to traditional measures (i.e., bibliometrics). They can help to identify the online attention received by articles, which might act as a further indicator for research assessment. One often mentioned advantage of these alternative indicators is, for example, that they appear much faster compared to citations. A large number of studies have explored altmetrics for different disciplines, but few studies have reported about altmetrics in the fields of Economics and Business Studies. Furthermore, no studies can be found so far that analyzed altmetrics within these disciplines with respect to libraries and information overload. Thus, this thesis explores opportunities for introducing altmetrics as new method for filtering relevant articles (in library portals) within the discipline of Economic and Business Studies literature. To achieve this objective, we have worked on four main aspects of investigating altmetrics and altmetrics data, respectively, of which the results can be used to fill the gap in this field of research. (1) We first highlight to what extent altmetric information from the two altmetric providers Mendeley and Altmetric.com is present within the journals of Economics and Business Studies. Based on the coverage, we demonstrate that altmetrics data are sparse in these disciplines, and when considering altmetrics data for real-world applications (e.g., in libraries), higher aggregation levels, such as journal level, can overcome their sparsity well. (2) We perform and discuss the correlations of citations on article and journal levels between different types and sources of altmetrics. We could show that Mendeley counts are positive and strongly correlated with citation counts on both article and journal levels, whereas other indicators such as Twitter counts and Altmetric Attention Score are significantly correlated only on journal level. With these correlations, we could suggest Mendeley counts for Economic and Business Studies journals/articles as an alternative indicator to citations. (3) In conjunction with the findings related to altmetrics in Economics and Business Studies journals, we discuss three use cases derived from three ZBW personas in terms of altmetrics. We investigate the use of altmetrics data for potential users with interests in new trends, social media platforms and journal rankings. (4) We investigated the behavior of economic researchers using a survey by exploring the usefulness of different altmetrics on journal level while they make decisions for selecting one article for reading. According to the user evaluation results, we demonstrate that altmetrics are not well known and understood by the economic community. However, this does not mean that these indicators are not helpful at all to economists. Instead, it brings forward the problem of how to introduce altmetrics to the economic community in the right way using which characteristics (e.g., as visible numbers attached at library records or behind the library’s relevance ranking system). Considering the aforementioned findings of this thesis, we can suggest several forms of presenting altmetric information in library portals, using EconBiz as the proof-of-concept, with the intention to assist both researchers and libraries to identify relevant journals or articles (e.g., highly mentioned online and recently published) for their need and to cope with the information overload

Collaborative yet independent: Information practices in the physical sciences

In many ways, the physical sciences are at the forefront of using digital tools and methods to work with information and data. However, the fields and disciplines that make up the physical sciences are by no means uniform, and physical scientists find, use, and disseminate information in a variety of ways. This report examines information practices in the physical sciences across seven cases, and demonstrates the richly varied ways in which physical scientists work, collaborate, and share information and data. This report details seven case studies in the physical sciences. For each case, qualitative interviews and focus groups were used to understand the domain. Quantitative data gathered from a survey of participants highlights different information strategies employed across the cases, and identifies important software used for research. Finally, conclusions from across the cases are drawn, and recommendations are made. This report is the third in a series commissioned by the Research Information Network (RIN), each looking at information practices in a specific domain (life sciences, humanities, and physical sciences). The aim is to understand how researchers within a range of disciplines find and use information, and in particular how that has changed with the introduction of new technologies

Technology transfer - A selected bibliography

Selected bibliography on technology transfe

Open Data, Grey Data, and Stewardship: Universities at the Privacy Frontier

As universities recognize the inherent value in the data they collect and hold, they encounter unforeseen challenges in stewarding those data in ways that balance accountability, transparency, and protection of privacy, academic freedom, and intellectual property. Two parallel developments in academic data collection are converging: (1) open access requirements, whereby researchers must provide access to their data as a condition of obtaining grant funding or publishing results in journals; and (2) the vast accumulation of 'grey data' about individuals in their daily activities of research, teaching, learning, services, and administration. The boundaries between research and grey data are blurring, making it more difficult to assess the risks and responsibilities associated with any data collection. Many sets of data, both research and grey, fall outside privacy regulations such as HIPAA, FERPA, and PII. Universities are exploiting these data for research, learning analytics, faculty evaluation, strategic decisions, and other sensitive matters. Commercial entities are besieging universities with requests for access to data or for partnerships to mine them. The privacy frontier facing research universities spans open access practices, uses and misuses of data, public records requests, cyber risk, and curating data for privacy protection. This paper explores the competing values inherent in data stewardship and makes recommendations for practice, drawing on the pioneering work of the University of California in privacy and information security, data governance, and cyber risk.Comment: Final published version, Sept 30, 201

Conceptualizations of language errors, standards, norms and nativeness in English for research publication purposes: An analysis of journal submission guidelines

Adherence to standards in English for research publication purposes (ERPP) can be a substantial barrier for second language (L2) writers and is an area of renewed debate in L2 writing research. This study presents a qualitative text analysis of author guidelines in 210 leading academic journals across 27 disciplines. It explores conceptualizations of language errors, standards, norms and nativeness in journal submission guidelines, and identifies key concepts related to so-called error-free writing. Findings indicate that most of the journal guidelines are inflexible in their acceptance of variant uses of English. Some guidelines state a requirement of meeting an unclear standard of good English, sometimes described as American or British English. Many guidelines specifically position L2 writers as deficient of native standards, which raises ethical considerations of access to publication in top journals. This study leads to a discussion of a need to re-conceptualize error-free writing in ERPP, and to decouple it from concepts such as nativeness. It focuses on a need to relax some author guidelines to encourage all authors to write using an English that can easily be understood by a broad, heterogeneous, global, and multilingual audience

Academic abstracts: a genre analysis

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro de Comunicação e Expressão. Programa de Pós-Graduação em Inglês e Literatura CorrespondenteResumos ("abstracts") em artigos acadêmicos são importantes meios de disseminação do conhecimento científico. Contudo, existem pouquíssimos estudos relativos a organização discursiva de resumos. Além disso, as normas disponíveis sobre a caracterização de resumos pouco ajudam na produção desse tipo de texto. O presente trabalho investiga a organização discursiva real de 94 resumos em três periódicos de maior circulação entre pesquisadores de Lingüística Aplicada. Através da análise de movimentos ("moves"), o estudo revelou que resumos seguem um padrão de 5 movimentos, a saber: o Movimento 1 motiva o leitor à pesquisa, indicando não só a área ou assunto mas também brechas em pesquisas anteriores; o Movimento 2 apresenta a pesquisa per se, descrevendo características principais e/ou objetivos; o Movimento 3 dispõe a metodologia utilizada; o Movimento 4 revela os resultados mais importantes; e o Movimento 5 discute o significado do estudo, apontando conclusões e/ou recomendações. Evidências sugerem que há discrepância entre prática e norma. O padrão proposto serve enquanto recurso pedagógico que auxilia pesquisadores a escreverem resumos mais comunicativos e assim contribuirem para avanços em sua comunidade científica. Recomenda-se a análise de gênero como uma abordagem valiosa para a descrição e explicação do discurso

Open Access Publishing: A Literature Review

Within the context of the Centre for Copyright and New Business Models in the Creative Economy (CREATe) research scope, this literature review investigates the current trends, advantages, disadvantages, problems and solutions, opportunities and barriers in Open Access Publishing (OAP), and in particular Open Access (OA) academic publishing. This study is intended to scope and evaluate current theory and practice concerning models for OAP and engage with intellectual, legal and economic perspectives on OAP. It is also aimed at mapping the field of academic publishing in the UK and abroad, drawing specifically upon the experiences of CREATe industry partners as well as other initiatives such as SSRN, open source software, and Creative Commons. As a final critical goal, this scoping study will identify any meaningful gaps in the relevant literature with a view to developing further research questions. The results of this scoping exercise will then be presented to relevant industry and academic partners at a workshop intended to assist in further developing the critical research questions pertinent to OAP

Rehabilitation/Recovery Journals

Comprehensive list of rehabilitation/recovery journal