Exploring gap filling as a cheaper alternative to reading comprehension questionnaires when evaluating machine translation for gisting

A popular application of machine translation (MT) is gisting: MT is consumed as is to make sense of text in a foreign language. Evaluation of the usefulness of MT for gisting is surprisingly uncommon. The classical method uses reading comprehension questionnaires (RCQ), in which informants are asked to answer professionally-written questions in their language about a foreign text that has been machine-translated into their language. Recently, gap-filling (GF), a form of cloze testing, has been proposed as a cheaper alternative to RCQ. In GF, certain words are removed from reference translations and readers are asked to fill the gaps left using the machine-translated text as a hint. This paper reports, for thefirst time, a comparative evaluation, using both RCQ and GF, of translations from multiple MT systems for the same foreign texts, and a systematic study on the effect of variables such as gap density, gap-selection strategies, and document context in GF. The main findings of the study are: (a) both RCQ and GF clearly identify MT to be useful, (b) global RCQ and GF rankings for the MT systems are mostly in agreement, (c) GF scores vary very widely across informants, making comparisons among MT systems hard, and (d) unlike RCQ, which is framed around documents, GF evaluation can be framed at the sentence level. These findings support the use of GF as a cheaper alternative to RCQ

Measuring comprehension and perception of neural machine translated texts : a pilot study

In this paper we compare the results of reading comprehension tests on both human translated and raw (unedited) machine translated texts. We selected three texts of the English Machine Translation Evaluation version (CREG-MT-eval) of the Corpus of Reading Comprehension Exercises (CREG), for which we produced three different translations: a manual translation and two automatic translations generated by two state-of-the-art neural machine translation engines, viz. DeepL and Google Translate. The experiment was conducted via a SurveyMonkey questionnaire, which 99 participants filled in. Participants were asked to read the translation very carefully after which they had to answer the comprehension questions without having access to the translated text. Apart from assessing comprehension, we posed additional questions to get information on the participants’ perception of the machine translations. The results show that 74% of the participants can tell whether a translation was produced by a human or a machine. Human translations received the best overall clarity scores, but the reading comprehension tests provided much less unequivocal results. The errors that bother readers most relate to grammar, sentence length, level of idiomaticity and incoherence

Bridging Web 4.0 and Education 4.0 For Next Generation User Training in ERP Adoption

This study addresses the critical issue of user comprehension and application within the sphere of cloudbased Enterprise Resource Planning (ERP) systems, a recurrent challenge exacerbated by the intricate nature of these systems. To bridge the existing gaps in training methodologies, a novel paradigm that synergizes Web 4.0 and Education 4.0 modules with traditional ERP systems is proposed. This innovative framework ushers in a paradigm shift in ERP adoption strategies, promising a marked enhancement in user interaction and efficiency. Rigorous qualitative evaluations, conducted with expert panels and potential end-users, provided robust validation of the framework's transformative potential in the realm of user training for ERP systems. This pioneering approach not only makes a substantial academic contribution by reframing the perception of ERP systems but also holds a significant practical value in ameliorating the user experience with cloud-based ERP systems. In essence, the adoption of a Web 4.0-oriented approach in user training heralds a revolutionary shift in ERP adoption strategies, setting a solid foundation for future explorations in this domain

JTEC panel report on machine translation in Japan

The goal of this report is to provide an overview of the state of the art of machine translation (MT) in Japan and to provide a comparison between Japanese and Western technology in this area. The term 'machine translation' as used here, includes both the science and technology required for automating the translation of text from one human language to another. Machine translation is viewed in Japan as an important strategic technology that is expected to play a key role in Japan's increasing participation in the world economy. MT is seen in Japan as important both for assimilating information into Japanese as well as for disseminating Japanese information throughout the world. Most of the MT systems now available in Japan are transfer-based systems. The majority of them exploit a case-frame representation of the source text as the basis of the transfer process. There is a gradual movement toward the use of deeper semantic representations, and some groups are beginning to look at interlingua-based systems

Prediction, interpolation and extrapolation of drilling data with Deep Learning

Directional drilling is an established technology within the petroleum industry. In this traditionally conservative and risk averse environment application of artificial intelligence encounters difficulties at various stages of the process. Additionally, the unique nature of drilling data makes deployment of the off-the-shelf algorithms problematic on multiple levels. An artificial neural network with a custom architecture is explored in this dissertation; it is combining recurrent elements as well as traditional artificial neurons to fully utilize information in both the dynamic behaviour of the system, as well as data that is available in real time. This fit for purpose algorithm unlocks significant improvements to the traditional directional drilling technologies. To complement this novel architecture, research is presented that focuses on the data preparation. Methods are presented that tackle characteristic problems of the real-time drilling logs that make the incompatible data digestible for the machine learning. New algorithms were developed that allow to gauge the difference between the raw and the processed data. Simplifying the field deployment a method for on-the-job training is explored for the developed architecture, where no prior knowledge about the exact drilling system nor historical data is required. To aid the confidence in the presented methods a fit for purpose sensitivity analysis is investigated allowing to peek inside the data driven algorithm

MODERNIZATION OF THE MOCK CIRCULATORY LOOP: ADVANCED PHYSICAL MODELING, HIGH PERFORMANCE HARDWARE, AND INCORPORATION OF ANATOMICAL MODELS

A systemic mock circulatory loop plays a pivotal role as the in vitro assessment tool for left heart medical devices. The standard design employed by many research groups dates to the early 1970\u27s, and lacks the acuity needed for the advanced device designs currently being explored. The necessity to update the architecture of this in vitro tool has become apparent as the historical design fails to deliver the performance needed to simulate conditions and events that have been clinically identified as challenges for future device designs. In order to appropriately deliver the testing solution needed, a comprehensive evaluation of the functionality demanded must be understood. The resulting system is a fully automated systemic mock circulatory loop, inclusive of anatomical geometries at critical flow sections, and accompanying software tools to execute precise investigations of cardiac device performance. Delivering this complete testing solution will be achieved through three research aims: (1) Utilization of advanced physical modeling tools to develop a high fidelity computational model of the in vitro system. This model will enable control design of the logic that will govern the in vitro actuators, allow experimental settings to be evaluated prior to execution in the mock circulatory loop, and determination of system settings that replicate clinical patient data. (2) Deployment of a fully automated mock circulatory loop that allows for runtime control of all the settings needed to appropriately construct the conditions of interest. It is essential that the system is able to change set point on the fly; simulation of cardiovascular dynamics and event sequences require this functionality. The robustness of an automated system with incorporated closed loop control logic yields a mock circulatory loop with excellent reproducibility, which is essential for effective device evaluation. (3) Incorporating anatomical geometry at the critical device interfaces; ascending aorta and left atrium. These anatomies represent complex shapes; the flows present in these sections are complex and greatly affect device performance. Increasing the fidelity of the local flow fields at these interfaces delivers a more accurate representation of the device performance in vivo

What types of feedback enhance the effectiveness of self-explanation in a simulation-based learning environment?

In this research, self-explanation was prompted and feedback was supplied to help learners activate prior knowledge, detect misconceptions, and replace unscientific mental models with correct scientific models. The research investigated the effects of two types of tutor feedback on learning and conceptual change in a simulation inquiry environment: Elaborative feedback incorporated tutor explanation and knowledge of results feedback provided only confirmation or disconfirmation of learners’ statements. Sixty-eight undergraduate students, with low prior knowledge in the physics of waves, were randomly assigned to receive either (a) self-explanation prompts with no feedback (NF), (b) self-explanation prompts with knowledge of results feedback (KRF), and (c) self-explanation prompts with elaborative feedback (EF). A pretest-posttest design was used to investigate participants’ knowledge gain and conceptual change resulting from learning tasks they performed by interacting with a physics simulation and explaining what they observed. The simulation, learning tasks, and knowledge tests focused on five fundamental principles of wave physics, four of which are often subject to misconceptions. Chi-square tests of association followed by pairwise Fisher’s exact test comparisons revealed elaborative feedback was advantageous, but only for two of the four concepts prone to persistent misconception – the mechanism of sound propagation and the medium-speed relationship. The findings suggest that prompting learners to self-explain can be sufficient for learning, but only for concepts whose acquisition is not hindered by persistent misconceptions. For concepts prone to such misconceptions, elaborative feedback may be necessary for understanding phenomena at deep structural levels. It is proposed that self-explanation combined with elaborative feedback may be a highly effective instructional strategy across many scientific domains, especially in the context of simulation-based inquiry learning