736,552 research outputs found
A Survey on Evaluation Metrics for Backchannel Prediction Models
In this paper we give an overview of the evaluation metrics used to measure the performance of backchannel prediction models. Both objective and subjective evaluation metrics are discussed. The survey shows that almost every backchannel prediction model is evaluated with a different evaluation metric. This makes comparison between developed models unreliable, even beside the other variables in play, such as different corpora, language, conversational setting, amount of data and/or definition of the term backchannel
Quantum techniques using continuous variables of light
We present schemes for the generation and evaluation of continuous variable
entanglement of bright optical beams and give a brief overview of the variety
of optical techniques and quantum communication applications on this basis. A
new entanglement-based quantum interferometry scheme with bright beams is
suggested. The performance of the presented schemes is independent of the
relative interference phase which is advantageous for quantum communication
applications.Comment: 11 pages, 5 figures; minor correction, accepted versio
User Experience Evaluation in BCI: Filling the Gap
Brain-computer interface (BCI) systems can improve the user experience (UX) when used in entertainment technologies. Improved UX can enhance user acceptance, improve quality of life and also increase the system performance of a BCI system. Therefore, the evaluation of UX is essential in BCI research. However, BCI systems are generally evaluated according to the system aspect only so there is no methodology to evaluate UX in BCI systems. This paper gives an overview of such methods from the human-computer interaction field and discusses their possible uses in BCI research
TRECVID 2008 - goals, tasks, data, evaluation mechanisms and metrics
The TREC Video Retrieval Evaluation (TRECVID) 2008 is a TREC-style video analysis and retrieval evaluation, the goal of which remains to promote progress in content-based exploitation of digital video via open, metrics-based evaluation. Over the last 7 years this effort has yielded a
better understanding of how systems can effectively accomplish such processing and how one can reliably benchmark their performance. In 2008, 77 teams (see Table 1) from various research organizations --- 24 from
Asia, 39 from Europe, 13 from North America, and 1 from Australia --- participated in one or more of five tasks: high-level feature extraction, search (fully automatic, manually assisted, or interactive), pre-production video (rushes) summarization, copy detection, or surveillance event detection. The copy detection and surveillance event detection tasks are being run for the first time in TRECVID.
This paper presents an overview of TRECVid in 2008
LSHTC: A Benchmark for Large-Scale Text Classification
LSHTC is a series of challenges which aims to assess the performance of
classification systems in large-scale classification in a a large number of
classes (up to hundreds of thousands). This paper describes the dataset that
have been released along the LSHTC series. The paper details the construction
of the datsets and the design of the tracks as well as the evaluation measures
that we implemented and a quick overview of the results. All of these datasets
are available online and runs may still be submitted on the online server of
the challenges
A Dual Digraph Approach for Leaderless Atomic Broadcast (Extended Version)
Many distributed systems work on a common shared state; in such systems,
distributed agreement is necessary for consistency. With an increasing number
of servers, these systems become more susceptible to single-server failures,
increasing the relevance of fault-tolerance. Atomic broadcast enables
fault-tolerant distributed agreement, yet it is costly to solve. Most practical
algorithms entail linear work per broadcast message. AllConcur -- a leaderless
approach -- reduces the work, by connecting the servers via a sparse resilient
overlay network; yet, this resiliency entails redundancy, limiting the
reduction of work. In this paper, we propose AllConcur+, an atomic broadcast
algorithm that lifts this limitation: During intervals with no failures, it
achieves minimal work by using a redundancy-free overlay network. When failures
do occur, it automatically recovers by switching to a resilient overlay
network. In our performance evaluation of non-failure scenarios, AllConcur+
achieves comparable throughput to AllGather -- a non-fault-tolerant distributed
agreement algorithm -- and outperforms AllConcur, LCR and Libpaxos both in
terms of throughput and latency. Furthermore, our evaluation of failure
scenarios shows that AllConcur+'s expected performance is robust with regard to
occasional failures. Thus, for realistic use cases, leveraging redundancy-free
distributed agreement during intervals with no failures improves performance
significantly.Comment: Overview: 24 pages, 6 sections, 3 appendices, 8 figures, 3 tables.
Modifications from previous version: extended the evaluation of AllConcur+
with a simulation of a multiple datacenters deploymen
- …