No-propagate algorithm for linear responses of random chaotic systems

We develop the no-propagate algorithm for sampling the linear response of random dynamical systems, which are non-uniform hyperbolic deterministic systems perturbed by noise with smooth density. We first derive a Monte-Carlo type formula and then the algorithm, which is different from the ensemble (stochastic gradient) algorithms, finite-element algorithms, and fast-response algorithms; it does not involve the propagation of vectors or covectors, and only the density of the noise is differentiated, so the formula is not cursed by gradient explosion, dimensionality, or non-hyperbolicity. We demonstrate our algorithm on a tent map perturbed by noise and a chaotic neural network with 51 layers $\times$ 9 neurons. By itself, this algorithm approximates the linear response of non-hyperbolic deterministic systems, with an additional error proportional to the noise. We also discuss the potential of using this algorithm as a part of a bigger algorithm with smaller error

Adjoint shadowing for backpropagation in hyperbolic chaos

For both discrete-time and continuous-time hyperbolic chaos, we introduce the adjoint shadowing operator $\mathcal{S}$ acting on covector fields. We show that $\mathcal{S}$ can be equivalently defined as: (a) $\mathcal{S}$ is the adjoint of the linear shadowing operator; (b) $\mathcal{S}$ is given by a `split then propagate' expansion formula; (c) $\mathcal{S}(\omega)$ is the only bounded inhomogeneous adjoint solution of $\omega$. By (a), $\mathcal{S}$ adjointly expresses the shadowing contribution, the most significant part of the linear response, where the linear response is the derivative of the long-time statistics with respect to parameters. By (b), $\mathcal{S}$ also expresses the other part of the linear response, the unstable contribution. By (c), $\mathcal{S}$ can be efficiently computed by the nonintrusive shadowing algorithm, which is similar to the conventional backpropagation algorithm. For continuous-time cases, we additionally show that the linear response admits a well-defined decomposition into shadowing and unstable contributions.Comment: 20 page

Equivariant divergence formula for chaotic flows

We prove the equivariant divergence formula for the axiom A flow attractors, which is a recursive formula for perturbation of transfer operators of physical measures along center-unstable manifolds. Hence the linear response acquires an `ergodic theorem', which means that it can be sampled by recursively computing only $2u$ many vectors on one orbit, where $u$ is the unstable dimension.Comment: comments are welcome

Sensitivity analysis of chaotic systems using a frequency-domain shadowing approach

We present a frequency-domain method for computing the sensitivities of time-averaged quantities of chaotic systems with respect to input parameters. Such sensitivities cannot be computed by conventional adjoint analysis tools, because the presence of positive Lyapunov exponents leads to exponential growth of the adjoint variables. The proposed method is based on the well established least-square shadowing (LSS) approach [1], that formulates the evaluation of sensitivities as an optimisation problem, thereby avoiding the exponential growth of the solution. All existing formulations of LSS (and its variants) are in the time domain. In the present paper, we reformulate the LSS method in the frequency (Fourier) space using harmonic balancing. The resulting system is closed using periodicity. The new method is tested on the Kuramoto-Sivashinsky system and the results match with those obtained using the standard time-domain formulation. The storage and computing requirements of the direct solution grow rapidly with the size of the system. To mitigate these requirements, we propose a resolvent-based iterative approach that needs much less storage. Application to the Kuramoto-Sivashinsky system gave accurate results with low computational cost. Truncating the large frequencies with small energy content from the harmonic balancing operator did not affect the accuracy of the computed sensitivities. Further work is needed to assess the performance and scalability of the proposed method