Sensitivity analysis of chaotic systems using a frequency-domain shadowing approach

We present a frequency-domain method for computing the sensitivities of time-averaged quantities of chaotic systems with respect to input parameters. Such sensitivities cannot be computed by conventional adjoint analysis tools, because the presence of positive Lyapunov exponents leads to exponential growth of the adjoint variables. The proposed method is based on the least-square shadowing (LSS) approach [1], that formulates the evaluation of sensitivities as an optimisation problem, thereby avoiding the exponential growth of the solution. However, all existing formulations of LSS (and its variants) are in the time domain and the computational cost scales with the number of positive Lyapunov exponents. In the present paper, we reformulate the LSS method in the Fourier space using harmonic balancing. The new method is tested on the Kuramoto-Sivashinski system and the results match with those obtained using the standard time-domain formulation. Although the cost of the direct solution is independent of the number of positive Lyapunov exponents, storage and computing requirements grow rapidly with the size of the system. To mitigate these requirements, we propose a resolvent-based iterative approach that needs much less storage. Application to the Kuramoto-Sivashinski system gave accurate results with very low computational cost. The method is applicable to large systems and paves the way for application of the resolvent-based shadowing approach to turbulent flows. Further work is needed to assess its performance and scalability

Adjoint shadowing for backpropagation in hyperbolic chaos

For both discrete-time and continuous-time hyperbolic chaos, we introduce the adjoint shadowing operator $\mathcal{S}$ acting on covector fields. We show that $\mathcal{S}$ can be equivalently defined as: (a) $\mathcal{S}$ is the adjoint of the linear shadowing operator; (b) $\mathcal{S}$ is given by a `split then propagate' expansion formula; (c) $\mathcal{S}(\omega)$ is the only bounded inhomogeneous adjoint solution of $\omega$. By (a), $\mathcal{S}$ adjointly expresses the shadowing contribution, the most significant part of the linear response, where the linear response is the derivative of the long-time statistics with respect to parameters. By (b), $\mathcal{S}$ also expresses the other part of the linear response, the unstable contribution. By (c), $\mathcal{S}$ can be efficiently computed by the nonintrusive shadowing algorithm, which is similar to the conventional backpropagation algorithm. For continuous-time cases, we additionally show that the linear response admits a well-defined decomposition into shadowing and unstable contributions.Comment: 20 page

Efficient sensitivity analysis of chaotic systems and applications to control and data assimilation

Sensitivity analysis is indispensable for aeronautical engineering applications that require optimisation, such as flow control and aircraft design. The adjoint method is the standard approach for sensitivity analysis, but it cannot be used for chaotic systems. This is due to the high sensitivity of the system trajectory to input perturbations; a characteristic of many turbulent systems. Although the instantaneous outputs are sensitive to input perturbations, the sensitivities of time-averaged outputs are well-defined for uniformly hyperbolic systems, but existing methods to compute them cannot be used. Recently, a set of alternative approaches based on the shadowing property of dynamical systems was proposed to compute sensitivities. These approaches are computationally expensive, however. In this thesis, the Multiple Shooting Shadowing (MSS) [1] approach is used, and the main aim is to develop computational tools to allow for the implementation of MSS to large systems. The major contributor to the cost of MSS is the solution of a linear matrix system. The matrix has a large condition number, and this leads to very slow convergence rates for existing iterative solvers. A preconditioner was derived to suppress the condition number, thereby accelerating the convergence rate. It was demonstrated that for the chaotic 1D Kuramoto Sivashinsky equation (KSE), the rate of convergence was almost independent of the #DOF and the trajectory length. Most importantly, the developed solution method relies only on matrix-vector products. The adjoint version of the preconditioned MSS algorithm was then coupled with a gradient descent method to compute a feedback control matrix for the KSE. The adopted formulation allowed all matrix elements to be computed simultaneously. Within a single iteration, a stabilising matrix was computed. Comparisons with standard linear quadratic theory (LQR) showed remarkable similarities (but also some differences) in the computed feedback control kernels. A preconditioned data assimilation algorithm was then derived for state estimation purposes. The preconditioner was again shown to accelerate the rate of convergence significantly. Accurate state estimations were computed for the Lorenz system.Open Acces

Sensitivity analysis of chaotic systems using a frequency-domain shadowing approach

We present a frequency-domain method for computing the sensitivities of time-averaged quantities of chaotic systems with respect to input parameters. Such sensitivities cannot be computed by conventional adjoint analysis tools, because the presence of positive Lyapunov exponents leads to exponential growth of the adjoint variables. The proposed method is based on the well established least-square shadowing (LSS) approach [1], that formulates the evaluation of sensitivities as an optimisation problem, thereby avoiding the exponential growth of the solution. All existing formulations of LSS (and its variants) are in the time domain. In the present paper, we reformulate the LSS method in the frequency (Fourier) space using harmonic balancing. The resulting system is closed using periodicity. The new method is tested on the Kuramoto-Sivashinsky system and the results match with those obtained using the standard time-domain formulation. The storage and computing requirements of the direct solution grow rapidly with the size of the system. To mitigate these requirements, we propose a resolvent-based iterative approach that needs much less storage. Application to the Kuramoto-Sivashinsky system gave accurate results with low computational cost. Truncating the large frequencies with small energy content from the harmonic balancing operator did not affect the accuracy of the computed sensitivities. Further work is needed to assess the performance and scalability of the proposed method

Deep Learning Closure of the Navier-Stokes Equations for Transition-Continuum Flows

The predictive accuracy of the Navier-Stokes equations is known to degrade at the limits of the continuum assumption, thereby necessitating expensive and often highly approximate solutions to the Boltzmann equation. While tractable in one spatial dimension, their high dimensionality makes multi-dimensional Boltzmann calculations impractical for all but canonical configurations. It is therefore desirable to augment the Navier-Stokes equations in these regimes. We present an application of a deep learning method to extend the validity of the Navier-Stokes equations to the transition-continuum flows. The technique encodes the missing physics via a neural network, which is trained directly from Boltzmann solutions. While standard DL methods can be considered ad-hoc due to the absence of underlying physical laws, at least in the sense that the systems are not governed by known partial differential equations, the DL framework leverages the a-priori known Boltzmann physics while ensuring that the trained model is consistent with the Navier-Stokes equations. The online training procedure solves adjoint equations, constructed using algorithmic differentiation, which efficiently provide the gradient of the loss function with respect to the learnable parameters. The model is trained and applied to predict stationary, one-dimensional shock thickness in low-pressure argon

Adjoint Methods as Design Tools in Thermoacoustics

In a thermoacoustic system, such as a flame in a combustor, heat release oscillations couple with acoustic pressure oscillations. If the heat release is sufficiently in phase with the pressure, these oscillations can grow, sometimes with catastrophic consequences. Thermoacoustic instabilities are still one of the most challenging problems faced by gas turbine and rocket motor manufacturers. Thermoacoustic systems are characterized by many parameters to which the stability may be extremely sensitive. However, often only few oscillation modes are unstable. Existing techniques examine how a change in one parameter affects all (calculated) oscillation modes, whether unstable or not. Adjoint techniques turn this around: They accurately and cheaply compute how each oscillation mode is affected by changes in all parameters. In a system with a million parameters, they calculate gradients a million times faster than finite difference methods. This review paper provides: (i) the methodology and theory of stability and adjoint analysis in thermoacoustics, which is characterized by degenerate and nondegenerate nonlinear eigenvalue problems; (ii) physical insight in the thermoacoustic spectrum, and its exceptional points; (iii) practical applications of adjoint sensitivity analysis to passive control of existing oscillations, and prevention of oscillations with ad hoc design modifications; (iv) accurate and efficient algorithms to perform uncertainty quantification of the stability calculations; (v) adjoint-based methods for optimization to suppress instabilities by placing acoustic dampers, and prevent instabilities by design modifications in the combustor's geometry; (vi) a methodology to gain physical insight in the stability mechanisms of thermoacoustic instability (intrinsic sensitivity); and (vii) in nonlinear periodic oscillations, the prediction of the amplitude of limit cycles with weakly nonlinear analysis, and the theoretical framework to calculate the sensitivity to design parameters of limit cycles with adjoint Floquet analysis. To show the robustness and versatility of adjoint methods, examples of applications are provided for different acoustic and flame models, both in longitudinal and annular combustors, with deterministic and probabilistic approaches. The successful application of adjoint sensitivity analysis to thermoacoustics opens up new possibilities for physical understanding, control and optimization to design safer, quieter, and cleaner aero-engines. The versatile methods proposed can be applied to other multiphysical and multiscale problems, such as fluid–structure interaction, with virtually no conceptual modification.</jats:p