Pushing Stochastic Gradient towards Second-Order Methods -- Backpropagation Learning with Transformations in Nonlinearities

Recently, we proposed to transform the outputs of each hidden neuron in a multi-layer perceptron network to have zero output and zero slope on average, and use separate shortcut connections to model the linear dependencies instead. We continue the work by firstly introducing a third transformation to normalize the scale of the outputs of each hidden neuron, and secondly by analyzing the connections to second order optimization methods. We show that the transformations make a simple stochastic gradient behave closer to second-order optimization methods and thus speed up learning. This is shown both in theory and with experiments. The experiments on the third transformation show that while it further increases the speed of learning, it can also hurt performance by converging to a worse local optimum, where both the inputs and outputs of many hidden neurons are close to zero.Comment: 10 pages, 5 figures, ICLR201

The Impact of Student Editing and Error-based Instruction on Writing in a Fourth-Grade Classroom

The purpose of the action research project was to determine the impact of self- and peer-editing and error-based instruction on student writing. The study took place in a fourth-grade classroom. There were 22 students who participated in the study. Data was collected through a writing self-reflection, weekly pre- and post- writing samples, an editing checklist, and an error tally sheet. Data was also collected through teacher observations. The results indicated that self- and peer-editing strategies along with error-based instruction had a positive impact on student writing. Although errors continued to persist within the students’ writing at the end of the study, the students were more aware of their writing skills. The implication is that writing needs to be viewed by students and teachers as a continuous process, analyzing and reanalyzing students’ work to guide instruction and practice. Keywords: student writing, self-editing, peer-editing, error-based instructio

Attitude determination of the spin-stabilized Project Scanner spacecraft

Attitude determination of spin-stabilized spacecraft using star mapping techniqu

Response of finite-time particle detectors in non-inertial frames and curved spacetime

The response of the Unruh-DeWitt type monopole detectors which were coupled to the quantum field only for a finite proper time interval is studied for inertial and accelerated trajectories, in the Minkowski vacuum in (3+1) dimensions. Such a detector will respond even while on an inertial trajctory due to the transient effects. Further the response will also depend on the manner in which the detector is switched on and off. We consider the response in the case of smooth as well as abrupt switching of the detector. The former case is achieved with the aid of smooth window functions whose width, $T$, determines the effective time scale for which the detector is coupled to the field. We obtain a general formula for the response of the detector when a window function is specified, and work out the response in detail for the case of gaussian and exponential window functions. A detailed discussion of both $T \rightarrow 0$ and $T \rightarrow \infty$ limits are given and several subtlities in the limiting procedure are clarified. The analysis is extended for detector responses in Schwarzschild and de-Sitter spacetimes in (1+1) dimensions.Comment: 29 pages, normal TeX, figures appended as postscript file, IUCAA Preprint # 23/9

Deep Neural Networks - A Brief History

Introduction to deep neural networks and their history.Comment: 14 pages, 14 figure

Identifying Nearby UHECR Accelerators using UHE (and VHE) Photons

Ultra-high energy photons (UHE, E > 10^19 eV) are inevitably produced during the propagation of 10^20 eV protons in extragalactic space. Their short interaction lengths (<20 Mpc) at these energies, combined with the impressive sensitivity of the Pierre Auger Observatory detector to these particles, makes them an ideal probe of nearby ultra-high-energy cosmic ray (UHECR) sources. We here discuss the particular case of photons from a single nearby (within 30 Mpc) source in light of the possibility that such an object might be responsible for several of the UHECR events published by the Auger collaboration. We demonstrate that the photon signal accompanying a cluster of a few > 6x10^19 eV UHECRs from such a source should be detectable by Auger in the near future. The detection of these photons would also be a signature of a light composition of the UHECRs from the nearby source.Comment: 4 pages, 2 figures, accepted for publication in PR