Bolstering Stochastic Gradient Descent with Model Building

Stochastic gradient descent method and its variants constitute the core optimization algorithms that achieve good convergence rates for solving machine learning problems. These rates are obtained especially when these algorithms are fine-tuned for the application at hand. Although this tuning process can require large computational costs, recent work has shown that these costs can be reduced by line search methods that iteratively adjust the stepsize. We propose an alternative approach to stochastic line search by using a new algorithm based on forward step model building. This model building step incorporates second-order information that allows adjusting not only the stepsize but also the search direction. Noting that deep learning model parameters come in groups (layers of tensors), our method builds its model and calculates a new step for each parameter group. This novel diagonalization approach makes the selected step lengths adaptive. We provide convergence rate analysis, and experimentally show that the proposed algorithm achieves faster convergence and better generalization in well-known test problems. More precisely, SMB requires less tuning, and shows comparable performance to other adaptive methods

Timed Implementation Relations for the Distributed Test Architecture

In order to test systems that have physically distributed interfaces, called ports, we might use a distributed approach in which there is a separate tester at each port. If the testers do not synchronise during testing then we cannot always determine the relative order of events observed at different ports and this leads to new notions of correctness that have been described using corresponding implementation relations. We study the situation in which each tester has a local clock and timestamps its observations. If we know nothing about how the local clocks relate then this does not affect the implementation relation while if the local clocks agree exactly then we can reconstruct the sequence of observations made. In practice, however, we are likely to be between these extremes: the local clocks will not agree exactly but we have some information regarding how they can differ. We start by assuming that a local tester interacts synchronously with the corresponding port of the system under test and then extend this to the case where communications can be asynchronous, considering both the first-in-first-out (FIFO) case and the non-FIFO case. The new implementation relations are stronger than implementation relations for distributed testing that do not use timestamps but still reflect the distributed nature of observations. This paper explores these alternatives and derives corresponding implementation relations

Distinguishing sequences for partially specified FSMs

Distinguishing Sequences (DSs) are used inmany Finite State Machine (FSM) based test techniques. Although Partially Specified FSMs (PSFSMs) generalise FSMs, the computational complexity of constructing Adaptive and Preset DSs (ADSs/PDSs) for PSFSMs has not been addressed. This paper shows that it is possible to check the existence of an ADS in polynomial time but the corresponding problem for PDSs is PSPACE-complete. We also report on the results of experiments with benchmarks and over 8 * 106 PSFSMs. © 2014 Springer International Publishing

Generating a checking sequence with a minimum number of reset transitions

Given a finite state machine M, a checking sequence is an input sequence that is guaranteed to lead to a failure if the implementation under test is faulty and has no more states than M. There has been much interest in the automated generation of a short checking sequence from a finite state machine. However, such sequences can contain reset transitions whose use can adversely affect both the cost of applying the checking sequence and the effectiveness of the checking sequence. Thus, we sometimes want a checking sequence with a minimum number of reset transitions rather than a shortest checking sequence. This paper describes a new algorithm for generating a checking sequence, based on a distinguishing sequence, that minimises the number of reset transitions used.This work was supported in part by Leverhulme Trust grant number F/00275/D, Testing State Based Systems, Natural Sciences and Engineering Research Council (NSERC) of Canada grant number RGPIN 976, and Engineering and Physical Sciences Research Council grant number GR/R43150, Formal Methods and Testing (FORTEST)

Search for Θ+(1540)\Theta^+(1540) pentaquark in high statistics measurement of γp→Kˉ0K+n\gamma p \to \bar K^0 K^+ n at CLAS

The exclusive reaction $\gamma p \to \bar K^0 K^+ n$ was studied in the photon energy range between 1.6-3.8 GeV searching for evidence of the exotic baryon $\Theta^+(1540)\to nK^+$. The decay to $nK^+$ requires the assignment of strangeness $S=+1$ to any observed resonance. Data were collected with the CLAS detector at the Thomas Jefferson National Accelerator Facility corresponding to an integrated luminosity of 70 $pb^{-1}$. No evidence for the $\Theta^+$ pentaquark was found. Upper limits were set on the production cross section as function of center-of-mass angle and $nK^+$ mass. The 95% CL upper limit on the total cross section for a narrow resonance at 1540 MeV was found to be 0.8 nb.Comment: Submitted to Physical Review Letter

Q^2 Dependence of the S_{11}(1535) Photocoupling and Evidence for a P-wave resonance in eta electroproduction

New cross sections for the reaction $ep \to e'\eta p$ are reported for total center of mass energy $W$=1.5--2.3 GeV and invariant squared momentum transfer $Q^2$=0.13--3.3 GeV$^2$. This large kinematic range allows extraction of new information about response functions, photocouplings, and $\eta N$ coupling strengths of baryon resonances. A sharp structure is seen at $W\sim$ 1.7 GeV. The shape of the differential cross section is indicative of the presence of a $P$-wave resonance that persists to high $Q^2$. Improved values are derived for the photon coupling amplitude for the $S_{11}$(1535) resonance. The new data greatly expands the $Q^2$ range covered and an interpretation of all data with a consistent parameterization is provided.Comment: 31 pages, 9 figure

Measurement of Deeply Virtual Compton Scattering with a Polarized Proton Target

The longitudinal target-spin asymmetry A_UL for the exclusive electroproduction of high energy photons was measured for the first time in p(e,e'p\gamma). The data have been accumulated at Jefferson Lab with the CLAS spectrometer using 5.7 GeV electrons and a longitudinally polarized NH_3 target. A significant azimuthal angular dependence was observed, resulting from the interference of the Deeply Virtual Compton Scattering and Bethe-Heitler processes. The amplitude of the sin(phi) moment is 0.252 +/- 0.042(stat) +/- 0.020(sys). Theoretical calculations are in good agreement with the magnitude and the kinematic dependence of the target-spin asymmetry, which is sensitive to the generalized parton distributions H and H-tilde.Comment: Modified text slightly, added reference

First measurement of coherent ϕ\phi-meson photoproduction on deuteron at low energies

The cross section and decay angular distributions for the coherent \phi meson photoproduction on the deuteron have been measured for the first time up to a squared four-momentum transfer t =(p_{\gamma}-p_{\phi})^2 =-2 GeV^2/c^2, using the CLAS detector at the Thomas Jefferson National Accelerator Facility. The cross sections are compared with predictions from a re-scattering model. In a framework of vector meson dominance, the data are consistent with the total \phi-N cross section \sigma_{\phi N} at about 10 mb. If vector meson dominance is violated, a larger \sigma_{\phi N} is possible by introducing larger t-slope for the \phi N \to \phi N process than that for the \gamma N \to \phi N process. The decay angular distributions of the \phi are consistent with helicity conservation.Comment: 6 page

Electron Scattering From High-Momentum Neutrons in Deuterium

We report results from an experiment measuring the semi-inclusive reaction $d(e,e'p_s)$ where the proton $p_s$ is moving at a large angle relative to the momentum transfer. If we assume that the proton was a spectator to the reaction taking place on the neutron in deuterium, the initial state of that neutron can be inferred. This method, known as spectator tagging, can be used to study electron scattering from high-momentum (off-shell) neutrons in deuterium. The data were taken with a 5.765 GeV electron beam on a deuterium target in Jefferson Laboratory's Hall B, using the CLAS detector. A reduced cross section was extracted for different values of final-state missing mass $W^{*}$, backward proton momentum $\vec{p}_{s}$ and momentum transfer $Q^{2}$. The data are compared to a simple PWIA spectator model. A strong enhancement in the data observed at transverse kinematics is not reproduced by the PWIA model. This enhancement can likely be associated with the contribution of final state interactions (FSI) that were not incorporated into the model. A ``bound neutron structure function'' $F_{2n}^{eff}$ was extracted as a function of $W^{*}$ and the scaling variable $x^{*}$ at extreme backward kinematics, where effects of FSI appear to be smaller. For $p_{s}>400$ MeV/c, where the neutron is far off-shell, the model overestimates the value of $F_{2n}^{eff}$ in the region of $x^{*}$ between 0.25 and 0.6. A modification of the bound neutron structure function is one of possible effects that can cause the observed deviation.Comment: 33 pages RevTeX, 9 figures, to be submitted to Phys. Rev. C. Fixed 1 Referenc