Long Short-Term Memory Spatial Transformer Network

Spatial transformer network has been used in a layered form in conjunction with a convolutional network to enable the model to transform data spatially. In this paper, we propose a combined spatial transformer network (STN) and a Long Short-Term Memory network (LSTM) to classify digits in sequences formed by MINST elements. This LSTM-STN model has a top-down attention mechanism profit from LSTM layer, so that the STN layer can perform short-term independent elements for the statement in the process of spatial transformation, thus avoiding the distortion that may be caused when the entire sequence is spatially transformed. It also avoids the influence of this distortion on the subsequent classification process using convolutional neural networks and achieves a single digit error of 1.6\% compared with 2.2\% of Convolutional Neural Network with STN layer

Anticipating Critical Transitions with Nonlinearity, Periodicity and Heterogeneity

Many natural and engineering systems may switch abruptly from one stable state to another due to a small perturbation to the system's state or a small change in the underlining conditions. In ecosystems, for example, extinctions of species or desertification can occur rapidly. Therefore, critical transitions can be dangerous to a number of systems, and it could be very beneficial if monitoring or early warning methods were available while the system is still in the healthy regime. The approach of critical transitions in many natural and engineering systems is accompanied by a phenomenon called critical slowing down. Theoretical and experimental studies have suggested that responses to small perturbations become increasingly slow when these systems are near critical transitions. Statistics such as variance, autocorrelation calculated from time series data have been proposed as early warning signals to anticipate the system's approach to a transition point. The problem of anticipating critical transitions becomes more complicated when other factors come into play. Factors such as nonlinearity, periodicity and heterogeneity can alter the behavior of the system, and thus affect the applicability of generic early warning signals. This thesis examines the effect of these factors on the critical transition of a system, and develops new data-driven approaches accordingly. To deal with and exploit the existence of nonlinearity in the system, recoveries from large instead of small perturbations are used to calculate the recovery rates of the system versus amplitudes. Under the circumstances of periodicity, recovery rates are calculated discretely via the Poincare section. Using experimental and computational data, we show that a combination of using recoveries from large perturbations and calculating recovery rates using the Poincare section can be highly effective in terms of anticipating critical transitions for systems with parametric resonance. Moreover, this thesis develops new early warning signals for spatially extended systems based on the eigenvalues of the covariance matrix. We mathematically show that the dominance of the largest eigenvalue of the covariance matrix can be used as an early warning signal by establishing the relationship between the eigenvalues of the covariance matrix and the eigenvalues of the force matrix. This new set of early warning signals are especially useful when the system has strong spatial heterogeneity. Lastly, this thesis investigates the influence of the choice of hyper-parameters, such as moving window size, sample rate, detrending methods, on the robustness of several early warning signals. General rules regarding data preparation and hypothesis testing are proposed.PHDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/145907/1/shychen_1.pd

Forecasting Bifurcation of Parametrically Excited Systems: Theory & Experiments

A system is parametrically excited when one or some of its coefficients vary with time. Parametric excitation can be observed in various engineered and physical systems. Many systems subject to parametric excitation exhibit critical transitions from one state to another as one or several of the system parameters change. Such critical transitions can either be caused by a change in the topological structure of the unforced system, or by synchronization between a natural mode of the system and the parameter variation. Forecasting bifurcations of parametrically excited systems before they occur is an active area of research both for engineered and natural systems. In particular, anticipating the distance to critical transitions, and predicting the state of the system after such transitions, remains a challenge, especially when there is an explicit time input to the system. In this work, a new model-less method is presented to address these problems based on monitoring transient recoveries from large perturbations in the pre-bifurcation regime. Recoveries are studied in a Poincare section to address the challenge caused by explicit time input. Numerical simulations and experimental results are provided to demonstrate the proposed method. In numerical simulation, a parametrically excited logistic equation and a parametrically excited Duffing oscillator are used to generate simulation data. These two types of systems show that the method can predict transitions induced by either bifurcation of the unforced system, or by parametric resonance. We further examine the robustness of the method to measurement and process noise by collecting recovery data from an electrical circuit system which exhibits parametric resonance as one of its parameters varies

Reducing the impact of non-ideal PRBS on microwave photonic random demodulators by low biasing the optical modulator via PRBS amplitude compression

A novel method for reducing the impact of non-ideal pseudo-random binary sequence (PRBS) on microwave photonic random demodulators (RDs) in a photonics-assisted compressed sensing (CS) system is proposed. Different from the commonly used method that switches the bias point of the optical modulator in the RD between two quadrature transmission points to mix the signal to be sampled and the PRBS, this method employs a PRBS with lower amplitude to low bias the optical modulator so that the impact of non-ideal PRBS on microwave photonic RDs can be greatly reduced by compressing the amplitude of non-ideal parts of the PRBS. An experiment is performed to verify the concept. The optical modulator is properly low-biased via PRBS amplitude compression. The data rate and occupied bandwidth of the PRBS are 500 Mb/s and 1 GHz, while the multi-tone signals with a maximum frequency of 100 MHz are sampled at an equivalent sampling rate of only 50 MSa/s. The results show that the reconstruction error can be reduced by up to 85%. The proposed method can significantly reduce the requirements for PRBS in RD-based photonics-assisted CS systems, providing a feasible solution for reducing the complexity and cost of system implementation.Comment: 9 pages, 5 figure

Kernel Fusion in Atomistic Spin Dynamics Simulations on Nvidia GPUs using Tensor Core

In atomistic spin dynamics simulations, the time cost of constructing the space- and time-displaced pair correlation function in real space increases quadratically as the number of spins $N$, leading to significant computational effort. The GEMM subroutine can be adopted to accelerate the calculation of the dynamical spin-spin correlation function, but the computational cost of simulating large spin systems ($>40000$ spins) on CPUs remains expensive. In this work, we perform the simulation on the graphics processing unit (GPU), a hardware solution widely used as an accelerator for scientific computing and deep learning. We show that GPUs can accelerate the simulation up to 25-fold compared to multi-core CPUs when using the GEMM subroutine on both. To hide memory latency, we fuse the element-wise operation into the GEMM kernel using $\mathtt{CUTLASS}$ that can improve the performance by 26% $\sim$ 33% compared to implementation based on $\mathtt{cuBLAS}$. Furthermore, we perform the on-the-fly calculation in the epilogue of the GEMM subroutine to avoid saving intermediate results on global memory, which makes the large-scale atomistic spin dynamics simulation feasible and affordable

Influences of tilted thin accretion disks on the optical appearance of hairy black holes in Horndeski gravity

Research on the optical appearance of black holes, both in general relativity and modified gravity, has been in full swing since the Event Horizon Telescope Collaboration announced photos of M87$^{*}$ and Sagittarius A$^{*}$. Nevertheless, limited attention has been given to the impact of tilted accretion disks on black hole images. This paper investigates the $230$ GHz images of non-rotating hairy black holes illuminated by tilted, thin accretion disks in Horndeski gravity with the aid of a ray tracing method. The results indicate that reducing the scalar hair parameter effectively diminishes image luminosity and extends both the critical curve and the inner shadow. This trend facilitates the differentiation between hairy black holes and Schwarzschild black holes. Furthermore, we observe that the inclination of the tilted accretion disk can mimic the observation angle, consequently affecting image brightness and the morphology of the inner shadow. In specific parameter spaces, the disk inclination has the ability to shift the position of the light spot in the images of hairy black holes. This finding may provide potential theoretical evidence for the formation of three flares at different positions in the Sagittarius A$^{*}$ image. Additionally, by examining the images of hairy black holes surrounded by two thin accretion disks, we report the obscuring effect of the accretion environment on the inner shadow of the black hole.Comment: 26 pages, 14 figure

Manipulating Electromagnetic Waves with Zero Index Materials

Zero-index material is a typical metamaterial with an effective zero refractive index, possessing a variety of exotic electromagnetic properties and particular functionalities. We have considered two kinds of zero-index materials with the first one a nearly matched zero index made of magnetic metamaterial and the second one a radially anisotropic zero index. The magnetic metamaterial-based systems are shown to be significant in wavefront engineering and flexibly tunable by an external magnetic field and a temperature field. The radially anisotropic zero-index-based systems can remarkably enhance the omnidirectional isotropic radiation by enclosing a line source and a dielectric particle within a shell configuration. The physical origin lies in that the dielectric particle effectively rescatters the trapped anisotropic higher order modes and converts them into the isotropic 0th order mode radiated outside the system. The case for the system with the loss is then examined and the energy compensation with a gain particle is also demonstrated

Study of phase transition of Potts model with DANN

A transfer learning method, domain adversarial neural network (DANN), is introduced to study the phase transition of two-dimensional q-state Potts model. With the DANN, we only need to choose a few labeled configurations automatically as input data, then the critical points can be obtained after training the algorithm. By an additional iterative process, the critical points can be captured to comparable accuracy to Monte Carlo simulations as we demonstrate it for q = 3, 5, 7 and 10. The type of phase transition (first or second-order) is also determined at the same time. Meanwhile, for the second-order phase transition at q = 3, we can calculate the critical exponent $\nu$ by data collapse. Furthermore, compared with the traditional supervised learning, the DANN is of higher accuracy with lower cost.Comment: 25 pages, 23 figure