Rethinking the Agreement in Human Evaluation Tasks

Human evaluations are broadly thought to be more valuable the higher the inter-annotator agreement. In this paper we examine this idea. We will describe our experiments and analysis within the area of Automatic Question Generation. Our experiments show how annotators diverge in language annotation tasks due to a range of ineliminable factors. For this reason, we believe that annotation schemes for natural language generation tasks that are aimed at evaluating language quality need to be treated with great care. In particular, an unchecked focus on reduction of disagreement among annotators runs the danger of creating generation goals that reward output that is more distant from, rather than closer to, natural human-like language. We conclude the paper by suggesting a new approach to the use of the agreement metrics in natural language generation evaluation tasks

Evaluation methodologies in Automatic Question Generation 2013-2018

In the last few years Automatic Question Generation (AQG) has attracted increasing interest. In this paper we survey the evaluation methodologies used in AQG. Based on a sample of 37 papers, our research shows that the systems’ development has not been accompanied by similar developments in the methodologies used for the systems’ evaluation. Indeed, in the papers we examine here, we find a wide variety of both intrinsic and extrinsic evaluation methodologies. Such diverse evaluation practices make it difficult to reliably compare the quality of different generation systems. Our study suggests that, given the rapidly increasing level of research in the area, a common framework is urgently needed to compare the performance of AQG systems and NLG systems more generally

European Acquisitions in the United States:Re-examining Olivetti-Underwood Fifty Years Later

While Italy's catch-up in the course of the 20th century has been nothing short of extraordinary, it has failed to produce a large number of global business players. Nonetheless, half a century ago an Italian company concluded what was at the time the largest-ever foreign takeover of a US company. The paper analyzes the Olivetti's acquisition of Underwood and frames it in the broader picture of the literature on the management and performance of foreign companies in the United States. We provide a historical narrative focused on three main issues: 1) head office control and subsidiary autonomy; 2) Olivetti's adaptation to the American business system; 3) the development of internal knowledge resources within the subsidiary. Lessons and implications are relevant for business historians and management scholars in general.FDI, acquisition, control, adaptation, knowledge resources, Olivetti, Underwood

Innovation and Foreign Technology in Italy,1861-2011

The paper explores the long run evolution of Italy’s performance in technological innovation as a function of international technology transfer, reconstructing the different phases and dimensions of Italian innovative activity, tracking the transfer of foreign technological knowledge through a number of channels, analysing the impact of imported technology. The study is based on a newly constructed dataset, over the 1861-2009 period, composed of variables related to: innovation activity performance; foreign technology transfer; domestic absorptive and innovative capability. The analysis highlights, also by econometric assessment, the significant contribution of foreign technology both to innovation activity results and to productivity growth. Differences across channels of technology transfer and historical phases emerge, also in connection with the evolution of human capital endowment and domestic innovative capacity. Machinery imports contributed positively both to innovation activity and to productivity growth; inward FDI contributed positively to productivity growth, but not to indigenous innovation activity; the accumulation of technical human capital fuelled both. In the long Italian Golden Age, for the first time the association of foreign technological knowledge with indigenous innovation processes strengthened productivity significantly. More recently instead the dismal productivity growth is negatively associated with formalised innovation activity under-performance and reduced imports of disembodied technologyItaly,Technology Transfer,Innovation,Absorptive Capability,Patenting

Asymmetries in top quark pair production at hadron colliders

We review the asymmetries in top quark pair production at the Tevatron and the LHC. We summarize the experimental measurements and the interpretations of a possible excess in terms of new physics. We also review other top quark properties-emphasizing effects related to the $t \bar t$ asymmetries-as well as other collider signals.Comment: RevTeX 40 pages. Final version to be published in Reviews of Modern Physics, with several addition

Forward-backward asymmetry in t¯t production

We present new measurements of the forward-backward asymmetry in t¯t production, performed with 5 fb−1 of Tevatron p¯p collisions at center of mass energy 1.96 TeV, recorded and analyzed at CDF. Significant inclusive asymmetries are observed in both the lepton+jets and the dilepton decay modes of the t¯t pair. In the dilepton mode, the asymmetry is observed in the reconstructed top rapidity, and in the lepton rapidity difference which is independent of any top reconstruction. In the lepton plus jets sample, the full reconstruction of the top kinematics is used to measure the dependence of the asymmetry on the rapidity difference Δy and the invariant mass Mt¯t of the t¯t system, and the asymmetry is found to be most significant at large rapidity and mass

On the Forward-Backward Asymmetry of Leptonic Decays of ttˉt\bar{t} at the Fermilab Tevatron

We report on a study of the measurement techniques used to determine the leptonic forward-backward asymmetry of top anti-top quark pairs in Tevatron experiments with a proton anti-proton initial state. Recently it was shown that a fit of the differential asymmetry as a function of $q_{l}\eta_{l}$ (where $q_{l}$ is the charge of the lepton from the cascade decay of the top quarks and $\eta_{l}$ is the final pseudorapidity of the lepton in the detector frame) to a hyperbolic tangent function can be used to extrapolate to the full leptonic asymmetry. We find this empirical method to well reproduce the results from current experiments, and present arguments as to why this is the case. We also introduce two more models, based on Gaussian functions, that better model the $q_{l}\eta_{l}$ distribution. With our better understanding, we find that the asymmetry is mainly determined by the shift of the mean of the $q_{l}\eta_{l}$ distribution, the main contribution to the inclusive asymmetry comes from the region around $|q_{l}\eta_{l}| = 1$, and the extrapolation from the detector-covered region to the inclusive asymmetry is stable via a multiplicative scale factor, giving us confidence in the previously reported experimental results.Comment: 26 pages, 12 figure

The Tev 2000 report: Electroweak physics with high luminosity at the Fermilab Tevatron

The addition of the Main Injector and the Antiproton Recycler to the Fermilab accelerator complex create the prospect of integrating samples in excess of 20 fb−120fb−1 at the Tevatron collider. The Tev2000 report is a study of the “High-PTPT” physics that may be realized with such data sets. We find a rich catalog of measurements, including characterization of the top quark, sensitivity to a light Higgs boson and low energy SUSY, and a precision electroweak program with sensitivity comparable and complementary to that of LEP-I. © 1997 American Institute of Physics.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/87490/2/95_1.pd

Seeing constructed realities : images and law in the contemporary American novel

In the new digital era, novelists have developed new tools to aid them levy political criticism against targets that have traditionally fallen outside the reach of the novel. By examining four contemporary American novels, I seek to examine how novelists use embedded photography to levy political criticism against systems and entities that rely on verbal language. My chief argument is that the ambiguity of language, when compounded with the ambiguity of photographs, forces readers into a mode of active interpretation which allows them to question concepts they usually take for granted. The hope is that by better understanding how these authors convey their messages we can better understand not just the systems they criticize, but build a model for future activist writing and criticism.Dr. Samuel Cohen, Thesis Supervisor.Includes bibliographical references (pages 60-64)