1,925 research outputs found
Robust Independent Component Analysis via Minimum Divergence Estimation
Independent component analysis (ICA) has been shown to be useful in many
applications. However, most ICA methods are sensitive to data contamination and
outliers. In this article we introduce a general minimum U-divergence framework
for ICA, which covers some standard ICA methods as special cases. Within the
U-family we further focus on the gamma-divergence due to its desirable property
of super robustness, which gives the proposed method gamma-ICA. Statistical
properties and technical conditions for the consistency of gamma-ICA are
rigorously studied. In the limiting case, it leads to a necessary and
sufficient condition for the consistency of MLE-ICA. This necessary and
sufficient condition is weaker than the condition known in the literature.
Since the parameter of interest in ICA is an orthogonal matrix, a geometrical
algorithm based on gradient flows on special orthogonal group is introduced to
implement gamma-ICA. Furthermore, a data-driven selection for the gamma value,
which is critical to the achievement of gamma-ICA, is developed. The
performance, especially the robustness, of gamma-ICA in comparison with
standard ICA methods is demonstrated through experimental studies using
simulated data and image data.Comment: 7 figure
Learning Multi-Level Information for Dialogue Response Selection by Highway Recurrent Transformer
With the increasing research interest in dialogue response generation, there
is an emerging branch formulating this task as selecting next sentences, where
given the partial dialogue contexts, the goal is to determine the most probable
next sentence. Following the recent success of the Transformer model, this
paper proposes (1) a new variant of attention mechanism based on multi-head
attention, called highway attention, and (2) a recurrent model based on
transformer and the proposed highway attention, so-called Highway Recurrent
Transformer. Experiments on the response selection task in the seventh Dialog
System Technology Challenge (DSTC7) show the capability of the proposed model
of modeling both utterance-level and dialogue-level information; the
effectiveness of each module is further analyzed as well
- …