Non-linear Learning for Statistical Machine Translation

Modern statistical machine translation (SMT) systems usually use a linear combination of features to model the quality of each translation hypothesis. The linear combination assumes that all the features are in a linear relationship and constrains that each feature interacts with the rest features in an linear manner, which might limit the expressive power of the model and lead to a under-fit model on the current data. In this paper, we propose a non-linear modeling for the quality of translation hypotheses based on neural networks, which allows more complex interaction between features. A learning framework is presented for training the non-linear models. We also discuss possible heuristics in designing the network structure which may improve the non-linear learning performance. Experimental results show that with the basic features of a hierarchical phrase-based machine translation system, our method produce translations that are better than a linear model.Comment: submitted to a conferenc

Embedded Lensing Time Delays, the Fermat Potential, and the Integrated Sachs-Wolfe Effect

We derive the Fermat potential for a spherically symmetric lens embedded in an FLRW cosmology and use it to investigate the late-time integrated Sachs-Wolfe (ISW) effect, i.e., secondary temperature fluctuations in the cosmic microwave background (CMB) caused by individual large scale clusters and voids. We present a simple analytical expression for the temperature fluctuation in the CMB across such a lens as a derivative of the lens' Fermat potential. This formalism is applicable to both linear and nonlinear density evolution scenarios, to arbitrarily large density contrasts, and to all open and closed background cosmologies. It is much simpler to use and makes the same predictions as conventional approaches. In this approach the total temperature fluctuation can be split into a time-delay part and an evolutionary part. Both parts must be included for cosmic structures that evolve and both can be equally important. We present very simple ISW models for cosmic voids and galaxy clusters to illustrate the ease of use of our formalism. We use the Fermat potentials of simple cosmic void models to compare predicted ISW effects with those recently extracted from WMAP and \emph{Planck} data by stacking large cosmic voids using the aperture photometry method. If voids in the local universe with large density contrasts are no longer evolving we find that the time delay contribution alone predicts values consistent with the measurements. However, we find that for voids still evolving linearly, the evolutionary contribution cancels a significant part of the time delay contribution and results in predicted signals that are much smaller than recently observed.Comment: 25 pages, 4 figures, ApJ in pres

A Simple Gravitational Lens Model For Cosmic Voids

We present a simple gravitational lens model to illustrate the ease of using the embedded lensing theory when studying cosmic voids. It confirms the previously used repulsive lensing models for deep voids. We start by estimating magnitude fluctuations and weak lensing shears of background sources lensed by large voids. We find that sources behind large ($\sim$$90\,\rm Mpc$) and deep voids (density contrast about $-0.9$) can be magnified or demagnified with magnitude fluctuations of up to $\sim$$0.05\,\rm mag$ and that the weak-lensing shear can be up to the $\sim$$10^{-2}$ level in the outer regions of large voids. Smaller or shallower voids produce proportionally smaller effects. We investigate the "wiggling" of the primary cosmic microwave background (CMB) temperature anisotropies caused by intervening cosmic voids. The void-wiggling of primary CMB temperature gradients is of the opposite sign to that caused by galaxy clusters. Only extremely large and deep voids can produce wiggling amplitudes similar to galaxy clusters, $\sim$$\rm 15\,\mu K$ by a large void of radius $\sim$$4^\circ$ and central density contrast $-0.9$ at redshift 0.5 assuming a CMB background gradient of $\sim$$\rm10\,\mu K\, arcmin^{-1}$. The dipole signal is spread over the entire void area, and not concentrated at the lens' center as it is for clusters. Finally we use our model to simulate CMB sky maps lensed by large cosmic voids. Our embedded theory can easily be applied to more complicated void models and used to study gravitational lensing of the CMB, to probe dark-matter profiles, to reduce the lensing-induced systematics in supernova Hubble diagrams, as well as study the integrated Sachs-Wolfe effect.Comment: 25 pages, 4 figures, ApJ accepte

Top-Rank Enhanced Listwise Optimization for Statistical Machine Translation

Pairwise ranking methods are the basis of many widely used discriminative training approaches for structure prediction problems in natural language processing(NLP). Decomposing the problem of ranking hypotheses into pairwise comparisons enables simple and efficient solutions. However, neglecting the global ordering of the hypothesis list may hinder learning. We propose a listwise learning framework for structure prediction problems such as machine translation. Our framework directly models the entire translation list's ordering to learn parameters which may better fit the given listwise samples. Furthermore, we propose top-rank enhanced loss functions, which are more sensitive to ranking errors at higher positions. Experiments on a large-scale Chinese-English translation task show that both our listwise learning framework and top-rank enhanced listwise losses lead to significant improvements in translation quality.Comment: Accepted to CONLL 201

Embedded Way of Responsible Innovation in ChatGPT

In the era of artificial intelligence, ChatGPT, as an advanced language model technology, has the potential for radical innovation. Despite its significant advantages, ChatGPT poses specific potential social and ethical issues. Therefore, we need responsible innovation to mitigate these risks and enable ChatGPT to benefit the global community truly. By embedding responsible innovation throughout the various stages of ChatGPT, we can ensure the practical realisation of public trust in governments and expectations from enterprises, thus achieving compliance and successful implementation. Through such a healthy development approach, we can ensure that ChatGPT positively impacts society and continues to foster its healthy growth