Adaptive Normalized Risk-Averting Training For Deep Neural Networks

This paper proposes a set of new error criteria and learning approaches, Adaptive Normalized Risk-Averting Training (ANRAT), to attack the non-convex optimization problem in training deep neural networks (DNNs). Theoretically, we demonstrate its effectiveness on global and local convexity lower-bounded by the standard $L_p$-norm error. By analyzing the gradient on the convexity index $\lambda$, we explain the reason why to learn $\lambda$ adaptively using gradient descent works. In practice, we show how this method improves training of deep neural networks to solve visual recognition tasks on the MNIST and CIFAR-10 datasets. Without using pretraining or other tricks, we obtain results comparable or superior to those reported in recent literature on the same tasks using standard ConvNets + MSE/cross entropy. Performance on deep/shallow multilayer perceptrons and Denoised Auto-encoders is also explored. ANRAT can be combined with other quasi-Newton training methods, innovative network variants, regularization techniques and other specific tricks in DNNs. Other than unsupervised pretraining, it provides a new perspective to address the non-convex optimization problem in DNNs.Comment: AAAI 2016, 0.39%~0.4% ER on MNIST with single 32-32-256-10 ConvNets, code available at https://github.com/cauchyturing/ANRA

A Nonparametric Approach to Pricing and Hedging Derivative Securities Via Learning Networks

We propose a nonparametric method for estimating the pricing formula of a derivative asset using learning networks. Although not a substitute for the more traditional arbitrage-based pricing formulas, network pricing formulas may be more accurate and computationally more efficient alternatives when the underlying asset's price dynamics are unknown, or when the pricing equation associated with no-arbitrage condition cannot be solved analytically. To assess the potential value of network pricing formulas, we simulate Black-Scholes option prices and show that learning networks can recover the Black-Scholes formula from a two-year training set of daily options prices, and that the resulting network formula can be used successfully to both price and delta-hedge options out-of-sample. For comparison, we estimate models using four popular methods: ordinary least squares, radial basis function networks, multilayer perceptron networks, and projection pursuit. To illustrate the practical relevance of our network pricing approach, we apply it to the pricing and delta-hedging of S&P 500 futures options from 1987 to 1991.

A Wireless Implantable System for Facilitating Gastrointestinal Motility.

Gastrointestinal (GI) electrical stimulation has been shown in several studies to be a potential treatment option for GI motility disorders. Despite the promising preliminary research progress, however, its clinical applicability and usability are still unknown and limited due to the lack of a miniaturized versatile implantable stimulator supporting the investigation of effective stimulation patterns for facilitating GI dysmotility. In this paper, we present a wireless implantable GI modulation system to fill this technology gap. The system consists of a wireless extraluminal gastrointestinal modulation device (EGMD) performing GI electrical stimulation, and a rendezvous device (RD) and a custom-made graphical user interface (GUI) outside the body to wirelessly power and configure the EGMD to provide the desired stimuli for modulating GI smooth muscle activities. The system prototype was validated in bench-top and in vivo tests. The GI modulation system demonstrated its potential for facilitating intestinal transit in the preliminary in vivo chronic study using porcine models

Dynamics of correlations due to a phase noisy laser

We analyze the dynamics of various kinds of correlations present between two initially entangled independent qubits, each one subject to a local phase noisy laser. We give explicit expressions of the relevant quantifiers of correlations for the general case of single-qubit unital evolution, which includes the case of a phase noisy laser. Although the light field is treated as classical, we find that this model can describe revivals of quantum correlations. Two different dynamical regimes of decay of correlations occur, a Markovian one (exponential decay) and a non-Markovian one (oscillatory decay with revivals) depending on the values of system parameters. In particular, in the non-Markovian regime, quantum correlations quantified by quantum discord show an oscillatory decay faster than that of classical correlations. Moreover, there are time regions where nonzero discord is present while entanglement is zero.Comment: 7 pages, 3 figures, accepted for publication in Phys. Scripta, special issue for CEWQO 2011 proceeding

Mrk 1419 - a new distance determination

Water vapor megamasers from the center of active galaxies provide a powerful tool to trace accretion disks at sub-parsec resolution and, through an entirely geometrical method, measure direct distances to galaxies up to 200 Mpc. The Megamaser Cosmology Project (MCP) is formed by a team of astronomers with the aim of identifying new maser systems, and mapping their emission at high angular resolution to determine their distance. Two types of observations are necessary to measure a distance: single-dish monitoring to measure the acceleration of gas in the disk, and sensitive VLBI imaging to measure the angular size of the disk, measure the rotation curve, and model radial displacement of the maser feature. The ultimate goal of the MCP is to make a precise measurement of H0 by measuring such distances to at least 10 maser galaxies in the Hubble flow. We present here the preliminary results from a new maser system, Mrk 1419. Through a model of the rotation from the systemic masers assuming a narrow ring, and combining these results with the acceleration measurement from the Green Bank Telescope, we determine a distance to Mrk 1419 of 81\pm10 Mpc. Given that the disk shows a significant warp that may not be entirely traced by our current observations, more sensitive observations and more sophisticated disk modeling will be essential to improve our distance estimation to this galaxy.Comment: 5 pages, 3 figures, to appear in the proceedings of IAU Symposium 287 "Cosmic Masers- from OH to Ho", in Stellenbosch, S

The Megamaser Cosmology Project. VII. Investigating disk physics using spectral monitoring observations

We use single-dish radio spectra of known 22 GHz H$_2$O megamasers, primarily gathered from the large dataset observed by the Megamaser Cosmology Project, to identify Keplerian accretion disks and to investigate several aspects of the disk physics. We test a mechanism for maser excitation proposed by Maoz & McKee (1998), whereby population inversion arises in gas behind spiral shocks traveling through the disk. Though the flux of redshifted features is larger on average than that of blueshifted features, in support of the model, the high-velocity features show none of the predicted systematic velocity drifts. We find rapid intra-day variability in the maser spectrum of ESO 558-G009 that is likely the result of interstellar scintillation, for which we favor a nearby ($D \approx 70$ pc) scattering screen. In a search for reverberation in six well-sampled sources, we find that any radially-propagating signal must be contributing $\lesssim$10% of the total variability. We also set limits on the magnetic field strengths in seven sources, using strong flaring events to check for the presence of Zeeman splitting. These limits are typically 200--300 mG ($1\sigma$), but our most stringent limits reach down to 73 mG for the galaxy NGC 1194.Comment: Accepted for publication in Ap

Adopting Robustness and Optimality in Fitting and Learning

We generalized a modified exponentialized estimator by pushing the robust-optimal (RO) index $\lambda$ to $-\infty$ for achieving robustness to outliers by optimizing a quasi-Minimin function. The robustness is realized and controlled adaptively by the RO index without any predefined threshold. Optimality is guaranteed by expansion of the convexity region in the Hessian matrix to largely avoid local optima. Detailed quantitative analysis on both robustness and optimality are provided. The results of proposed experiments on fitting tasks for three noisy non-convex functions and the digits recognition task on the MNIST dataset consolidate the conclusions.Comment: arXiv admin note: text overlap with arXiv:1506.0269

The Submillimeter Array

The Submillimeter Array (SMA), a collaborative project of the Smithsonian Astrophysical Observatory (SAO) and the Academia Sinica Institute of Astronomy and Astrophysics (ASIAA), has begun operation on Mauna Kea in Hawaii. A total of eight 6-m telescopes comprise the array, which will cover the frequency range of 180-900 GHz. All eight telescopes have been deployed and are operational. First scientific results utilizing the three receiver bands at 230, 345, and 690 GHz have been obtained and are presented in the accompanying papers.Comment: 10 pages, 4 figure