Quantitative evaluation of recall and precision of CAT Crawler, a search engine specialized on retrieval of Critically Appraised Topics

BACKGROUND: Critically Appraised Topics (CATs) are a useful tool that helps physicians to make clinical decisions as the healthcare moves towards the practice of Evidence-Based Medicine (EBM). The fast growing World Wide Web has provided a place for physicians to share their appraised topics online, but an increasing amount of time is needed to find a particular topic within such a rich repository. METHODS: A web-based application, namely the CAT Crawler, was developed by Singapore's Bioinformatics Institute to allow physicians to adequately access available appraised topics on the Internet. A meta-search engine, as the core component of the application, finds relevant topics following keyword input. The primary objective of the work presented here is to evaluate the quantity and quality of search results obtained from the meta-search engine of the CAT Crawler by comparing them with those obtained from two individual CAT search engines. From the CAT libraries at these two sites, all possible keywords were extracted using a keyword extractor. Of those common to both libraries, ten were randomly chosen for evaluation. All ten were submitted to the two search engines individually, and through the meta-search engine of the CAT Crawler. Search results were evaluated for relevance both by medical amateurs and professionals, and the respective recall and precision were calculated. RESULTS: While achieving an identical recall, the meta-search engine showed a precision of 77.26% (±14.45) compared to the individual search engines' 52.65% (±12.0) (p < 0.001). CONCLUSION: The results demonstrate the validity of the CAT Crawler meta-search engine approach. The improved precision due to inherent filters underlines the practical usefulness of this tool for clinicians

Relevance similarity: an alternative means to monitor information retrieval systems

BACKGROUND: Relevance assessment is a major problem in the evaluation of information retrieval systems. The work presented here introduces a new parameter, "Relevance Similarity", for the measurement of the variation of relevance assessment. In a situation where individual assessment can be compared with a gold standard, this parameter is used to study the effect of such variation on the performance of a medical information retrieval system. In such a setting, Relevance Similarity is the ratio of assessors who rank a given document same as the gold standard over the total number of assessors in the group. METHODS: The study was carried out on a collection of Critically Appraised Topics (CATs). Twelve volunteers were divided into two groups of people according to their domain knowledge. They assessed the relevance of retrieved topics obtained by querying a meta-search engine with ten keywords related to medical science. Their assessments were compared to the gold standard assessment, and Relevance Similarities were calculated as the ratio of positive concordance with the gold standard for each topic. RESULTS: The similarity comparison among groups showed that a higher degree of agreements exists among evaluators with more subject knowledge. The performance of the retrieval system was not significantly different as a result of the variations in relevance assessment in this particular query set. CONCLUSION: In assessment situations where evaluators can be compared to a gold standard, Relevance Similarity provides an alternative evaluation technique to the commonly used kappa scores, which may give paradoxically low scores in highly biased situations such as document repositories containing large quantities of relevant data

Semantic technologies: from niche to the mainstream of Web 3? A comprehensive framework for web Information modelling and semantic annotation

Context: Web information technologies developed and applied in the last decade have considerably changed the way web applications operate and have revolutionised information management and knowledge discovery. Social technologies, user-generated classification schemes and formal semantics have a far-reaching sphere of influence. They promote collective intelligence, support interoperability, enhance sustainability and instigate innovation. Contribution: The research carried out and consequent publications follow the various paradigms of semantic technologies, assess each approach, evaluate its efficiency, identify the challenges involved and propose a comprehensive framework for web information modelling and semantic annotation, which is the thesis’ original contribution to knowledge. The proposed framework assists web information modelling, facilitates semantic annotation and information retrieval, enables system interoperability and enhances information quality. Implications: Semantic technologies coupled with social media and end-user involvement can instigate innovative influence with wide organisational implications that can benefit a considerable range of industries. The scalable and sustainable business models of social computing and the collective intelligence of organisational social media can be resourcefully paired with internal research and knowledge from interoperable information repositories, back-end databases and legacy systems. Semantified information assets can free human resources so that they can be used to better serve business development, support innovation and increase productivity

Combining SOA and BPM Technologies for Cross-System Process Automation

This paper summarizes the results of an industry case study that introduced a cross-system business process automation solution based on a combination of SOA and BPM standard technologies (i.e., BPMN, BPEL, WSDL). Besides discussing major weaknesses of the existing, custom-built, solution and comparing them against experiences with the developed prototype, the paper presents a course of action for transforming the current solution into the proposed solution. This includes a general approach, consisting of four distinct steps, as well as specific action items that are to be performed for every step. The discussion also covers language and tool support and challenges arising from the transformation

Study on open science: The general state of the play in Open Science principles and practices at European life sciences institutes

Nowadays, open science is a hot topic on all levels and also is one of the priorities of the European Research Area. Components that are commonly associated with open science are open access, open data, open methodology, open source, open peer review, open science policies and citizen science. Open science may a great potential to connect and influence the practices of researchers, funding institutions and the public. In this paper, we evaluate the level of openness based on public surveys at four European life sciences institute

Promoting Statistical Practice and Collaboration in Developing Countries

"Rarely, but just often enough to rebuild hope, something happens to confound my pessimism about the recent unprecedented happenings in the world. This book is the most recent instance, and I think that all its readers will join me in rejoicing at the good it seeks to do. It is an example of the kind of international comity and collaboration that we could and should undertake to solve various societal problems. This book is a beautiful example of the power of the possible. [It] provides a blueprint for how the LISA 2020 model can be replicated in other fields. Civil engineers, or accountants, or nurses, or any other profession could follow this outline to share expertise and build capacity and promote progress in other countries. It also contains some tutorials for statistical literacy across several fields. The details would change, of course, but ideas are durable, and the generalizations seem pretty straightforward. This book shows every other profession where and how to stand in order to move the world. I urge every researcher to get a copy!" —David Banks from the Foreword Promoting Statistical Practice and Collaboration in Developing Countries provides new insights into the current issues and opportunities in international statistics education, statistical consulting, and collaboration, particularly in developing countries around the world. The book addresses the topics discussed in individual chapters from the perspectives of the historical context, the present state, and future directions of statistical training and practice, so that readers may fully understand the challenges and opportunities in the field of statistics and data science, especially in developing countries. Features • Reference point on statistical practice in developing countries for researchers, scholars, students, and practitioners • Comprehensive source of state-of-the-art knowledge on creating statistical collaboration laboratories within the field of data science and statistics • Collection of innovative statistical teaching and learning techniques in developing countries Each chapter consists of independent case study contributions on a particular theme that are developed with a common structure and format. The common goal across the chapters is to enhance the exchange of diverse educational and action-oriented information among our intended audiences, which include practitioners, researchers, students, and statistics educators in developing countries

Promoting Statistical Practice and Collaboration in Developing Countries

"Rarely, but just often enough to rebuild hope, something happens to confound my pessimism about the recent unprecedented happenings in the world. This book is the most recent instance, and I think that all its readers will join me in rejoicing at the good it seeks to do. It is an example of the kind of international comity and collaboration that we could and should undertake to solve various societal problems. This book is a beautiful example of the power of the possible. [It] provides a blueprint for how the LISA 2020 model can be replicated in other fields. Civil engineers, or accountants, or nurses, or any other profession could follow this outline to share expertise and build capacity and promote progress in other countries. It also contains some tutorials for statistical literacy across several fields. The details would change, of course, but ideas are durable, and the generalizations seem pretty straightforward. This book shows every other profession where and how to stand in order to move the world. I urge every researcher to get a copy!" —David Banks from the Foreword Promoting Statistical Practice and Collaboration in Developing Countries provides new insights into the current issues and opportunities in international statistics education, statistical consulting, and collaboration, particularly in developing countries around the world. The book addresses the topics discussed in individual chapters from the perspectives of the historical context, the present state, and future directions of statistical training and practice, so that readers may fully understand the challenges and opportunities in the field of statistics and data science, especially in developing countries. Features • Reference point on statistical practice in developing countries for researchers, scholars, students, and practitioners • Comprehensive source of state-of-the-art knowledge on creating statistical collaboration laboratories within the field of data science and statistics • Collection of innovative statistical teaching and learning techniques in developing countries Each chapter consists of independent case study contributions on a particular theme that are developed with a common structure and format. The common goal across the chapters is to enhance the exchange of diverse educational and action-oriented information among our intended audiences, which include practitioners, researchers, students, and statistics educators in developing countries

The impact of mathematics teaching efficacy on teachers’ pedagogical practices

This study explores the pedagogical practices of 167 Year 4 and 160 Year 8 New Zealand mathematics teachers who have different levels of mathematics teaching efficacy. Using data from the National Monitoring Study of Student Achievement 2013, the teacher questionnaire items believed to be the indicators of mathematics teaching efficacy were selected, represented by six items such as “I feel confident about teaching maths”. Then, low, mid, and high efficacious teachers were identified and compared to see how they differed with respect to their teaching profile and the frequency they used effective pedagogies when teaching mathematics (italicised below) (Anthony &amp; Walshaw, 2007). Twenty eight percent of Year 4 and 41% of Year 8 teachers had high mathematics teaching efficacy. Compared with the other teachers, teachers with high mathematics teaching efficacy were better able to provide an ethic of care in their classroom, they more frequently arranged their classrooms for learning to enable students to collaborate, and more frequently expected their students to communicate their thinking and debate ideas with others. They more frequently provided students with worthwhile mathematical tasks, they more frequently provided opportunities for their students to build on their own thinking, and to explore how new learning linked to or changed what they already knew. They more frequently expected their students to make mathematical connections by reflecting on their learning, to use multiple representations, and use ideas and skills from different curriculum areas

Building information modeling – A game changer for interoperability and a chance for digital preservation of architectural data?

Digital data associated with the architectural design-andconstruction process is an essential resource alongside -and even past- the lifecycle of the construction object it describes. Despite this, digital architectural data remains to be largely neglected in digital preservation research – and vice versa, digital preservation is so far neglected in the design-and-construction process. In the last 5 years, Building Information Modeling (BIM) has seen a growing adoption in the architecture and construction domains, marking a large step towards much needed interoperability. The open standard IFC (Industry Foundation Classes) is one way in which data is exchanged in BIM processes. This paper presents a first digital preservation based look at BIM processes, highlighting the history and adoption of the methods as well as the open file format standard IFC (Industry Foundation Classes) as one way to store and preserve BIM data