2,453 research outputs found
Spatial Blind Source Separation in the Presence of a Drift
Multivariate measurements taken at different spatial locations occur frequently in practice. Proper analysis of such data needs to consider not only dependencies on-sight but also dependencies in and in-between variables as a function of spatial separation. Spatial Blind Source Separation (SBSS) is a recently developed unsupervised statistical tool that deals with such data by assuming that the observable data is formed by a linear latent variable model. In SBSS the latent variable is assumed to be constituted by weakly stationary random fields which are uncorrelated. Such a model is appealing as further analysis can be carried out on the marginal distributions of the latent variables, interpretations are straightforward as the model is assumed to be linear, and not all components of the latent field might be of interest which acts as a form of dimension reduction. The weakly stationarity assumption of SBSS implies that the mean of the data is constant for all sample locations, which might be too restricting in practical applications. Therefore, an adaptation of SBSS that uses scatter matrices based on differences was recently suggested in the literature. In our contribution we formalize these ideas, suggest a novel adapted SBSS method and show its usefulness on synthetic data and illustrate its use in a real data application
Backpropagation Beyond the Gradient
Automatic differentiation is a key enabler of deep learning: previously, practitioners were limited to models
for which they could manually compute derivatives. Now, they can create sophisticated models with almost
no restrictions and train them using first-order, i. e. gradient, information. Popular libraries like PyTorch
and TensorFlow compute this gradient efficiently, automatically, and conveniently with a single line of
code. Under the hood, reverse-mode automatic differentiation, or gradient backpropagation, powers the
gradient computation in these libraries. Their entire design centers around gradient backpropagation.
These frameworks are specialized around one specific task—computing the average gradient in a mini-batch.
This specialization often complicates the extraction of other information like higher-order statistical moments
of the gradient, or higher-order derivatives like the Hessian. It limits practitioners and researchers to methods
that rely on the gradient. Arguably, this hampers the field from exploring the potential of higher-order
information and there is evidence that focusing solely on the gradient has not lead to significant recent
advances in deep learning optimization.
To advance algorithmic research and inspire novel ideas, information beyond the batch-averaged gradient
must be made available at the same level of computational efficiency, automation, and convenience.
This thesis presents approaches to simplify experimentation with rich information beyond the gradient
by making it more readily accessible. We present an implementation of these ideas as an extension to the
backpropagation procedure in PyTorch. Using this newly accessible information, we demonstrate possible use
cases by (i) showing how it can inform our understanding of neural network training by building a diagnostic
tool, and (ii) enabling novel methods to efficiently compute and approximate curvature information.
First, we extend gradient backpropagation for sequential feedforward models to Hessian backpropagation
which enables computing approximate per-layer curvature. This perspective unifies recently proposed block-
diagonal curvature approximations. Like gradient backpropagation, the computation of these second-order
derivatives is modular, and therefore simple to automate and extend to new operations.
Based on the insight that rich information beyond the gradient can be computed efficiently and at the
same time, we extend the backpropagation in PyTorch with the BackPACK library. It provides efficient and
convenient access to statistical moments of the gradient and approximate curvature information, often at a
small overhead compared to computing just the gradient.
Next, we showcase the utility of such information to better understand neural network training. We build
the Cockpit library that visualizes what is happening inside the model during training through various
instruments that rely on BackPACK’s statistics. We show how Cockpit provides a meaningful statistical
summary report to the deep learning engineer to identify bugs in their machine learning pipeline, guide
hyperparameter tuning, and study deep learning phenomena.
Finally, we use BackPACK’s extended automatic differentiation functionality to develop ViViT, an approach
to efficiently compute curvature information, in particular curvature noise. It uses the low-rank structure
of the generalized Gauss-Newton approximation to the Hessian and addresses shortcomings in existing
curvature approximations. Through monitoring curvature noise, we demonstrate how ViViT’s information
helps in understanding challenges to make second-order optimization methods work in practice.
This work develops new tools to experiment more easily with higher-order information in complex deep
learning models. These tools have impacted works on Bayesian applications with Laplace approximations,
out-of-distribution generalization, differential privacy, and the design of automatic differentia-
tion systems. They constitute one important step towards developing and establishing more efficient deep
learning algorithms
LIPIcs, Volume 251, ITCS 2023, Complete Volume
LIPIcs, Volume 251, ITCS 2023, Complete Volum
Floquet formulation of the dynamical Berry-phase approach to non-linear optics in extended systems
We present a Floquet scheme for the ab-initio calculation of nonlinear
optical properties in extended systems. This entails a reformulation of the
real-time approach based on the dynamical Berry-phase polarisation [Attaccalite
& Gr\"uning, PRB 88, 1-9 (2013)] and retains the advantage of being
non-perturbative in the electric field. The proposed method applies to
periodically-driven Hamiltonians and makes use of this symmetry to turn a
time-dependent problem into a self-consistent time-independent eigenvalue
problem. We implemented this Floquet scheme at the independent particle level
and compared it with the real-time approach. Our reformulation reproduces
real-time-calculated and order susceptibilities for a number
of bulk and two-dimensional materials, while reducing the associated
computational cost by one or two orders of magnitude
Locality and Exceptional Points in Pseudo-Hermitian Physics
Pseudo-Hermitian operators generalize the concept of Hermiticity. Included in this class of operators are the quasi-Hermitian operators, which define a generalization of quantum theory with real-valued measurement outcomes and unitary time evolution. This thesis is devoted to the study of locality in quasi-Hermitian theory, the symmetries and conserved quantities associated with non-Hermitian operators, and the perturbative features of pseudo-Hermitian matrices.
An implicit assumption of the tensor product model of locality is that the inner product factorizes with the tensor product. Quasi-Hermitian quantum theory generalizes the tensor product model by modifying the Born rule via a metric operator with nontrivial Schmidt rank. Local observable algebras and expectation values are examined in chapter 5. Observable algebras of two one-dimensional fermionic quasi-Hermitian chains are explicitly constructed. Notably, there can be spatial subsystems with no nontrivial observables. Despite devising a new framework for local quantum theory, I show that expectation values of local quasi-Hermitian observables can be equivalently computed as expectation values of Hermitian observables. Thus, quasi-Hermitian theories do not increase the values of nonlocal games set by Hermitian theories. Furthermore, Bell's inequality violations in quasi-Hermitian theories never exceed the Tsirelson bound of Hermitian quantum theory.
A perturbative feature present in pseudo-Hermitian curves which has no Hermitian counterpart is the exceptional point, a branch point in the set of eigenvalues. An original finding presented in section 2.6.3 is a correspondence between cusp singularities of algebraic curves and higher-order exceptional points. Eigensystems of one-dimensional lattice models admit closed-form expressions that can be used to explore the new features of non-Hermitian physics. One-dimensional lattice models with a pair of non Hermitian defect potentials with balanced gain and loss, Δ±iγ, are investigated in chapter 3. Conserved quantities and positive-definite metric operators are examined. When the defects are nearest neighbour, the entire spectrum simultaneously becomes complex when γ increases beyond a second-order exceptional point. When the defects are at the edges of the chain and the hopping amplitudes are 2-periodic, as in the Su-Schrieffer-Heeger chain, the PT-phase transition is dictated by the topological phase
of the system. In the thermodynamic limit, PT-symmetry spontaneously breaks in the topologically non-trivial phase due to the presence of edge states.
Chiral symmetry and representation theory are utilized in chapter 4 to derive large classes of pseudo-Hermitian operators with closed-form intertwining operators. These intertwining operators include positive-definite metric operators in the quasi-Hermitian case. The PT-phase transition is explicitly determined in a special case
Linear stability and numerical analysis of dipolar vortices and topographic flows
The linear stability and numerical analysis of geophysical flow patterns is carried out on the beta-plane in a quasigeostrophic approximation. We consider initial steady state dipoles in a one-and-a-half-layer model that are capable of zonal drift in either direction. Despite previous numerical works suggesting that eastward propagating dipoles are stable, our high resolution simulations identify the spontaneous symmetry breaking of weak dipoles over time. The evolution is associated with a growing critical mode with even symmetry about the zonal axis. On carrying out a linear stability analysis, the critical modes obtained share consistency with the numerical fields. In addition, both methods of analysis show that the linear growth rate is inversely proportional to the dipole intensity. Furthermore, the partner separation becomes more pronounced after the linear growth stage, suggesting that nonlinear effects play a pivotal role in the underlying dynamics. Beyond this, the dynamics of initially tilted dipoles and dipole-rider solutions are considered, while stronger dipoles are further analysed using the method of distillation.
Flows over sinusoidal bottom relief are considered in a two-layer model on the quasigeostrophic beta-plane. Fourier mode solutions are assumed for the layer-wise perturbation field in order to carry out a linear stability analysis, from which a coupled eigenproblem is derived between fluid columns for both zonal and meridional bottom irregularities. The presence of zonally oriented multiple ridges stabilises an otherwise unstable homogeneous zonal current with respect to increases in the number of ridges and ridge amplitude. Moreover, a bifurcation occurs in the unstable mode spectra and is dependent on the number of ridges. The critical eigenmodes in this case are found to be eddy chains of alternating sign, and these share remarkable resemblance with those obtained numerically. Meridionally oriented multiple ridges are also considered, but are found not to affect the maximum growth rate directly.Open Acces
Nonlinear and Linearized Analysis of Vibrations of Loaded Anisotropic Beam/Plate/Shell Structures
L'abstract è presente nell'allegato / the abstract is in the attachmen
Gyroscopic polynomials
Gyroscopic alignment of a fluid occurs when flow structures align with the
rotation axis. This often gives rise to highly spatially anisotropic columnar
structures that in combination with complex domain boundaries pose challenges
for efficient numerical discretizations and computations. We define gyroscopic
polynomials to be three-dimensional polynomials expressed in a coordinate
system that conforms to rotational alignment. We remap the original domain with
radius-dependent boundaries onto a right cylindrical or annular domain to
create the computational domain in this coordinate system. We find the volume
element expressed in gyroscopic coordinates leads naturally to a hierarchy of
orthonormal bases. We build the bases out of Jacobi polynomials in the vertical
and generalized Jacobi polynomials in the radial. Because these coordinates
explicitly conform to flow structures found in rapidly rotating systems the
bases represent fields with a relatively small number of modes. We develop the
operator structure for one-dimensional semi-classical orthogonal polynomials as
a building block for differential operators in the full three-dimensional
cylindrical and annular domains. The differential operators of generalized
Jacobi polynomials generate a sparse linear system for discretization of
differential operators acting on the gyroscopic bases. This enables efficient
simulation of systems with strong gyroscopic alignment
Beam scanning by liquid-crystal biasing in a modified SIW structure
A fixed-frequency beam-scanning 1D antenna based on Liquid Crystals (LCs) is designed for application in 2D scanning with lateral alignment. The 2D array environment imposes full decoupling of adjacent 1D antennas, which often conflicts with the LC requirement of DC biasing: the proposed design accommodates both. The LC medium is placed inside a Substrate Integrated Waveguide (SIW) modified to work as a Groove Gap Waveguide, with radiating slots etched on the upper broad wall, that radiates as a Leaky-Wave Antenna (LWA). This allows effective application of the DC bias voltage needed for tuning the LCs. At the same time, the RF field remains laterally confined, enabling the possibility to lay several antennas in parallel and achieve 2D beam scanning. The design is validated by simulation employing the actual properties of a commercial LC medium
- …