Machine translation evaluation resources and methods: a survey

We introduce the Machine Translation (MT) evaluation survey that contains both manual and automatic evaluation methods. The traditional human evaluation criteria mainly include the intelligibility, fidelity, fluency, adequacy, comprehension, and informativeness. The advanced human assessments include task-oriented measures, post-editing, segment ranking, and extended criteriea, etc. We classify the automatic evaluation methods into two categories, including lexical similarity scenario and linguistic features application. The lexical similarity methods contain edit distance, precision, recall, F-measure, and word order. The linguistic features can be divided into syntactic features and semantic features respectively. The syntactic features include part of speech tag, phrase types and sentence structures, and the semantic features include named entity, synonyms, textual entailment, paraphrase, semantic roles, and language models. The deep learning models for evaluation are very newly proposed. Subsequently, we also introduce the evaluation methods for MT evaluation including different correlation scores, and the recent quality estimation (QE) tasks for MT. This paper differs from the existing works\cite {GALEprogram2009, EuroMatrixProject2007} from several aspects, by introducing some recent development of MT evaluation measures, the different classifications from manual to automatic evaluation measures, the introduction of recent QE tasks of MT, and the concise construction of the content

Nonlinear heart rate variability features for real-life stress detection. Case study : students under stress due to university examination

Background: This study investigates the variations of Heart Rate Variability (HRV) due to a real-life stressor and proposes a classifier based on nonlinear features of HRV for automatic stress detection. Methods: 42 students volunteered to participate to the study about HRV and stress. For each student, two recordings were performed: one during an on-going university examination, assumed as a real-life stressor, and one after holidays. Nonlinear analysis of HRV was performed by using Poincaré Plot, Approximate Entropy, Correlation dimension, Detrended Fluctuation Analysis, Recurrence Plot. For statistical comparison, we adopted the Wilcoxon Signed Rank test and for development of a classifier we adopted the Linear Discriminant Analysis (LDA). Results: Almost all HRV features measuring heart rate complexity were significantly decreased in the stress session. LDA generated a simple classifier based on the two Poincaré Plot parameters and Approximate Entropy, which enables stress detection with a total classification accuracy, a sensitivity and a specificity rate of 90%, 86%, and 95% respectively. Conclusions: The results of the current study suggest that nonlinear HRV analysis using short term ECG recording could be effective in automatically detecting real-life stress condition, such as a university examination

K-Space at TRECVid 2007

In this paper we describe K-Space participation in TRECVid 2007. K-Space participated in two tasks, high-level feature extraction and interactive search. We present our approaches for each of these activities and provide a brief analysis of our results. Our high-level feature submission utilized multi-modal low-level features which included visual, audio and temporal elements. Specific concept detectors (such as Face detectors) developed by K-Space partners were also used. We experimented with different machine learning approaches including logistic regression and support vector machines (SVM). Finally we also experimented with both early and late fusion for feature combination. This year we also participated in interactive search, submitting 6 runs. We developed two interfaces which both utilized the same retrieval functionality. Our objective was to measure the effect of context, which was supported to different degrees in each interface, on user performance. The first of the two systems was a ‘shot’ based interface, where the results from a query were presented as a ranked list of shots. The second interface was ‘broadcast’ based, where results were presented as a ranked list of broadcasts. Both systems made use of the outputs of our high-level feature submission as well as low-level visual features

DCU-Symantec submission for the WMT 2012 quality estimation task

This paper describes the features and the machine learning methods used by Dublin City University (DCU) and SYMANTEC for the WMT 2012 quality estimation task. Two sets of features are proposed: one constrained, i.e. respecting the data limitation suggested by the workshop organisers, and one unconstrained, i.e. using data or tools trained on data that was not provided by the workshop organisers. In total, more than 300 features were extracted and used to train classifiers in order to predict the translation quality of unseen data. In this paper, we focus on a subset of our feature set that we consider to be relatively novel: features based on a topic model built using the Latent Dirichlet Allocation approach, and features based on source and target language syntax extracted using part-of-speech (POS) taggers and parsers. We evaluate nine feature combinations using four classification-based and four regression-based machine learning techniques

Normative Data and Minimally Detectable Change for Inner Retinal Layer Thicknesses Using a Semi-automated OCT Image Segmentation Pipeline

Neurodegenerative and neuroinflammatory diseases regularly cause optic nerve and retinal damage. Evaluating retinal changes using optical coherence tomography (OCT) in diseases like multiple sclerosis has thus become increasingly relevant. However, intraretinal segmentation, a necessary step for interpreting retinal changes in the context of these diseases, is not standardized and often requires manual correction. Here we present a semi-automatic intraretinal layer segmentation pipeline and establish normative values for retinal layer thicknesses at the macula, including dependencies on age, sex, and refractive error. Spectral domain OCT macular 3D volume scans were obtained from healthy participants using a Heidelberg Engineering Spectralis OCT. A semi-automated segmentation tool (SAMIRIX) based on an interchangeable third-party segmentation algorithm was developed and employed for segmentation, correction, and thickness computation of intraretinal layers. Normative data is reported froma 6mmEarly Treatment Diabetic Retinopathy Study (ETDRS) circle around the fovea. An interactive toolbox for the normative database allows surveying for additional normative data. We cross-sectionally evaluated data from218 healthy volunteers (144 females/74males, age 36.5 ± 12.3 years, range 18–69 years). Average macular thickness (MT) was 313.70 ± 12.02 μm, macular retinal nerve fiber layer thickness (mRNFL) 39.53 ± 3.57 μm, ganglion cell and inner plexiform layer thickness (GCIPL) 70.81 ± 4.87 μm, and inner nuclear layer thickness (INL) 35.93 ± 2.34 μm. All retinal layer thicknesses decreased with age. MT and GCIPL were associated with sex, with males showing higher thicknesses. Layer thicknesses were also positively associated with each other. Repeated-measurement reliability for the manual correction of automatic intraretinal segmentation results was excellent, with an intra-class correlation coefficient >0.99 for all layers. The SAMIRIX toolbox can simplify intraretinal segmentation in research applications, and the normative data application may serve as an expandable reference for studies, in which normative data cannot be otherwise obtained