258,911 research outputs found
Quality-Based Conditional Processing in Multi-Biometrics: Application to Sensor Interoperability
As biometric technology is increasingly deployed, it will be common to
replace parts of operational systems with newer designs. The cost and
inconvenience of reacquiring enrolled users when a new vendor solution is
incorporated makes this approach difficult and many applications will require
to deal with information from different sources regularly. These
interoperability problems can dramatically affect the performance of biometric
systems and thus, they need to be overcome. Here, we describe and evaluate the
ATVS-UAM fusion approach submitted to the quality-based evaluation of the 2007
BioSecure Multimodal Evaluation Campaign, whose aim was to compare fusion
algorithms when biometric signals were generated using several biometric
devices in mismatched conditions. Quality measures from the raw biometric data
are available to allow system adjustment to changing quality conditions due to
device changes. This system adjustment is referred to as quality-based
conditional processing. The proposed fusion approach is based on linear
logistic regression, in which fused scores tend to be log-likelihood-ratios.
This allows the easy and efficient combination of matching scores from
different devices assuming low dependence among modalities. In our system,
quality information is used to switch between different system modules
depending on the data source (the sensor in our case) and to reject channels
with low quality data during the fusion. We compare our fusion approach to a
set of rule-based fusion schemes over normalized scores. Results show that the
proposed approach outperforms all the rule-based fusion schemes. We also show
that with the quality-based channel rejection scheme, an overall improvement of
25% in the equal error rate is obtained.Comment: Published at IEEE Transactions on Systems, Man, and Cybernetics -
Part A: Systems and Human
Automatic Quality Estimation for ASR System Combination
Recognizer Output Voting Error Reduction (ROVER) has been widely used for
system combination in automatic speech recognition (ASR). In order to select
the most appropriate words to insert at each position in the output
transcriptions, some ROVER extensions rely on critical information such as
confidence scores and other ASR decoder features. This information, which is
not always available, highly depends on the decoding process and sometimes
tends to over estimate the real quality of the recognized words. In this paper
we propose a novel variant of ROVER that takes advantage of ASR quality
estimation (QE) for ranking the transcriptions at "segment level" instead of:
i) relying on confidence scores, or ii) feeding ROVER with randomly ordered
hypotheses. We first introduce an effective set of features to compensate for
the absence of ASR decoder information. Then, we apply QE techniques to perform
accurate hypothesis ranking at segment-level before starting the fusion
process. The evaluation is carried out on two different tasks, in which we
respectively combine hypotheses coming from independent ASR systems and
multi-microphone recordings. In both tasks, it is assumed that the ASR decoder
information is not available. The proposed approach significantly outperforms
standard ROVER and it is competitive with two strong oracles that e xploit
prior knowledge about the real quality of the hypotheses to be combined.
Compared to standard ROVER, the abs olute WER improvements in the two
evaluation scenarios range from 0.5% to 7.3%
Evaluating Temporal Persistence Using Replicability Measures
In real-world Information Retrieval (IR) experiments, the Evaluation
Environment (EE) is exposed to constant change. Documents are added, removed,
or updated, and the information need and the search behavior of users is
evolving. Simultaneously, IR systems are expected to retain a consistent
quality. The LongEval Lab seeks to investigate the longitudinal persistence of
IR systems, and in this work, we describe our participation. We submitted runs
of five advanced retrieval systems, namely a Reciprocal Rank Fusion (RRF)
approach, ColBERT, monoT5, Doc2Query, and E5, to both sub-tasks. Further, we
cast the longitudinal evaluation as a replicability study to better understand
the temporal change observed. As a result, we quantify the persistence of the
submitted runs and see great potential in this evaluation method.Comment: To be published in Proceedings of the Working Notes of CLEF 2023 -
Conference and Labs of the Evaluation Forum, Thessaloniki, Greece 18 - 21,
202
Recommended from our members
Algorithms for multi-modal human movement and behaviour monitoring
This thesis describes investigations into improvements in the field of automated people tracking using multi-modal infrared (IR) and visible image information. The research question posed is; “To what extent can infrared image information be used to improve visible light based human tracking systems?” Automated passive tracking of human subjects is an active research area which has been approached in many ways. Typical approaches include the segmentation of the foreground, the location of humans, model initialisation and subject tracking. Sensor reliability evaluation and fusion methods are also key research areas in multi-modal systems. Shifting illumination and shadows can cause issues with visible images when attempting to extract foreground regions. Images from thermal IR cameras, which use long-wavelength infrared (LWIR) sensors, demonstrate high invariance to illumination. It is shown that thermal IR images often provide superior foreground masks using pixel level statistical extraction techniques in many scenarios. Experiments are performed to determine if cues are present at the data level that may indicate the quality of the sensor as an input. Modality specific measures are proposed as possible indicators of sensor quality (determined by foreground extraction capability). A sensor and application specific method for scene evaluation is proposed, whereby sensor quality is measured at the pixel level. A neuro-fuzzy inference system is trained using the scene quality measures to assess a series of scenes and make a modality decision
On-line Metasearch, Pooling, and System Evaluation
This thesis presents a unified method for simultaneous solution of three problems in Information Retrieval--- metasearch (the fusion of ranked lists returned by retrieval systems to elicit improved performance), efficient system evaluation (the accurate evaluation of retrieval systems with small numbers of relevance judgements), and pooling or ``active sample selection (the selection of documents for manual judgement in order to develop sample pools of high precision or pools suitable for assessing system quality). The thesis establishes a unified theoretical framework for addressing these three problems and naturally generalizes their solution to the on-line context by incorporating feedback in the form of relevance judgements. The algorithm--- Rankhedge for on-line retrieval, metasearch and system evaluation--- is the first to address these three problems simultaneously and also to generalize their solution to the on-line context. Optimality of the Rankhedge algorithm is developed via Bayesian and maximum entropy interpretations. Results of the algorithm prove to be significantly superior to previous methods when tested over a range of TREC (Text REtrieval Conference) data. In the absence of feedback, the technique equals or exceeds the performance of benchmark metasearch algorithms such as CombMNZ and Condorcet. The technique then dramatically improves on this performance during the on-line metasearch process. In addition, the technique generates pools of documents which include more relevant documents and produce more accurate system evaluations than previous techniques. The thesis includes an information-theoretic examination of the original Hedge algorithm as well as its adaptation to the context of ranked lists. The work also addresses the concept of information-theoretic similarity within the Rankhedge context and presents a method for decorrelating the predictor set to improve worst case performance. Finally, an information-theoretically optimal method for probabilistic ``active sampling is presented with possible application to a broad range of practical and theoretical contexts
Fast computation of the performance evaluation of biometric systems: application to multibiometric
The performance evaluation of biometric systems is a crucial step when
designing and evaluating such systems. The evaluation process uses the Equal
Error Rate (EER) metric proposed by the International Organization for
Standardization (ISO/IEC). The EER metric is a powerful metric which allows
easily comparing and evaluating biometric systems. However, the computation
time of the EER is, most of the time, very intensive. In this paper, we propose
a fast method which computes an approximated value of the EER. We illustrate
the benefit of the proposed method on two applications: the computing of non
parametric confidence intervals and the use of genetic algorithms to compute
the parameters of fusion functions. Experimental results show the superiority
of the proposed EER approximation method in term of computing time, and the
interest of its use to reduce the learning of parameters with genetic
algorithms. The proposed method opens new perspectives for the development of
secure multibiometrics systems by speeding up their computation time.Comment: Future Generation Computer Systems (2012
- …