Intra-assessor consistency in question answering

In this paper we investigate the consistency of answer assessment in a complex question answering task examining features of assessor consistency, types of answers and question type

Overview of the CLEF-2005 cross-language speech retrieval track

The task for the CLEF-2005 cross-language speech retrieval track was to identify topically coherent segments of English interviews in a known-boundary condition. Seven teams participated, performing both monolingual and cross-language searches of ASR transcripts, automatically generated metadata, and manually generated metadata. Results indicate that monolingual search technology is sufficiently accurate to be useful for some purposes (the best mean average precision was 0.18) and cross-language searching yielded results typical of those seen in other applications (with the best systems approximating monolingual mean average precision)

A review of agricultural waste activated carbon and effect on adsorption parameters

Activated carbon is widely used in industrial wastewater treatment operation because of the effectiveness in removing the dye. As the conventional activated carbon on market nowadays is expensive, lots of research was done on agricultural materials to replace the activated carbon. Sugarcane bagasse is listed as one of the agricultural material alternatives. This review compiles the characterisation of sugarcane bagasse and several agricultural wastes-based activated carbon including proximate analysis, ultimate analysis, and activated carbon surface characteristic used to remove dye in textile wastewater. Nevertheless, the adsorption parameters are the main focus as it affecting the removal of dye. The expansion review regarding activated carbon performance implies that dye removal efficiency depends on the amount of adsorbent dosage, contact time, pH solution, dye concentration applied during the adsorption process. This review shows the need of thoughtful information on adsorption parameters with an expanded list of various types of agricultural-based activated carbon and various dye removal

Science Models as Value-Added Services for Scholarly Information Systems

The paper introduces scholarly Information Retrieval (IR) as a further dimension that should be considered in the science modeling debate. The IR use case is seen as a validation model of the adequacy of science models in representing and predicting structure and dynamics in science. Particular conceptualizations of scholarly activity and structures in science are used as value-added search services to improve retrieval quality: a co-word model depicting the cognitive structure of a field (used for query expansion), the Bradford law of information concentration, and a model of co-authorship networks (both used for re-ranking search results). An evaluation of the retrieval quality when science model driven services are used turned out that the models proposed actually provide beneficial effects to retrieval quality. From an IR perspective, the models studied are therefore verified as expressive conceptualizations of central phenomena in science. Thus, it could be shown that the IR perspective can significantly contribute to a better understanding of scholarly structures and activities.Comment: 26 pages, to appear in Scientometric

Evaluating epistemic uncertainty under incomplete assessments

The thesis of this study is to propose an extended methodology for laboratory based Information Retrieval evaluation under incomplete relevance assessments. This new methodology aims to identify potential uncertainty during system comparison that may result from incompleteness. The adoption of this methodology is advantageous, because the detection of epistemic uncertainty - the amount of knowledge (or ignorance) we have about the estimate of a system's performance - during the evaluation process can guide and direct researchers when evaluating new systems over existing and future test collections. Across a series of experiments we demonstrate how this methodology can lead towards a finer grained analysis of systems. In particular, we show through experimentation how the current practice in Information Retrieval evaluation of using a measurement depth larger than the pooling depth increases uncertainty during system comparison

GeoCLEF 2007: the CLEF 2007 cross-language geographic information retrieval track overview

GeoCLEF ran as a regular track for the second time within the Cross Language Evaluation Forum (CLEF) 2007. The purpose of GeoCLEF is to test and evaluate cross-language geographic information retrieval (GIR): retrieval for topics with a geographic specification. GeoCLEF 2007 consisted of two sub tasks. A search task ran for the third time and a query classification task was organized for the first. For the GeoCLEF 2007 search task, twenty-five search topics were defined by the organizing groups for searching English, German, Portuguese and Spanish document collections. All topics were translated into English, Indonesian, Portuguese, Spanish and German. Several topics in 2007 were geographically challenging. Thirteen groups submitted 108 runs. The groups used a variety of approaches. For the classification task, a query log from a search engine was provided and the groups needed to identify the queries with a geographic scope and the geographic components within the local queries

Assessment of shoulder active range of motion in prone versus supine:A reliability and concurrent validity study

Background: As swimming and surfing are prone dominant sports, it would be more sport specific to assess shoulder active range of motion in this position. Objectives: To determine the reliability of the inclinometer and HALO© for assessing shoulder active range of motion in supine and prone and the concurrent validity of the HALO©. Concurrent validity is based on the comparison of the HALO© and inclinometer. To determine if active range of motion (AROM) differences exists between prone and supine when assessing shoulder internal (IR) and external rotation (ER). Design: The design included clinical measurement, reliability and validity. Methods: Thirty shoulders (mean age = 26.8 years) without pathology were evaluated. Measurements were taken in supine and prone with both an inclinometer and HALO© device. Results: Active ER ROM in prone was significantly higher than in supine when using both devices. Intra-rater reliability (within and between session) intraclass correlation coefficient (ICC) values ranged between 0.82–0.99 for both devices in supine and prone. An ICC test revealed a significant (p \u3c 0.01) correlation for both devices in IR and ER movements (ICC3,1 = 0.87 and ICC3,1 = 0.72), respectively. Conclusion: This study has shown prone assessment of active ER and IR ROM to be a reliable and appropriate method for prone dominant athletes (swimmers and surfers). In this study greater ER ROM was achieved in prone compared to supine. This finding highlights the importance of standardizing the test position for initial and follow up assessments. Furthermore the HALO© and inclinometer have been shown to be reliable tools that show good concurrent validity

A novel method for subjective picture quality assessment and further studies of HDTV formats

This is the author's accepted manuscript. The final published article is available from the link below. Copyright @ IEEE 2008.This paper proposes a novel method for the assessment of picture quality, called triple stimulus continuous evaluation scale (TSCES), to allow the direct comparison of different HDTV formats. The method uses an upper picture quality anchor and a lower picture quality anchor with defined impairments. The HDTV format under test is evaluated in a subjective comparison with the upper and lower anchors. The method utilizes three displays in a particular vertical arrangement. In an initial series of tests with the novel method, the HDTV formats 1080p/50,1080i/25, and 720p/50 were compared at various bit-rates and with seven different content types on three identical 1920 times 1080 pixel displays. It was found that the new method provided stable and consistent results. The method was tested with 1080p/50,1080i/25, and 720p/50 HDTV images that had been coded with H.264/AVC High profile. The result of the assessment was that the progressive HDTV formats found higher appreciation by the assessors than the interlaced HDTV format. A system chain proposal is given for future media production and delivery to take advantage of this outcome. Recommendations for future research conclude the paper