Interpreting multi-stable behaviour in input-driven recurrent neural networks

Recurrent neural networks (RNNs) are computational models inspired by the brain. Although RNNs stand out as state-of-the-art machine learning models to solve challenging tasks as speech recognition, handwriting recognition, language translation, and others, they are plagued by the so-called vanishing/exploding gradient issue. This prevents us from training RNNs with the aim of learning long term dependencies in sequential data. Moreover, a problem of interpretability affects these models, known as the ``black-box issue'' of RNNs. We attempt to open the black box by developing a mechanistic interpretation of errors occurring during the computation. We do this from a dynamical system theory perspective, specifically building on the notion of Excitable Network Attractors. Our methodology is effective at least for those tasks where a number of attractors and a switching pattern between them must be learned. RNNs can be seen as massively large nonlinear dynamical systems driven by external inputs. When it comes to analytically investigate RNNs, often in the literature the input-driven property is neglected or dropped in favour of tight constraints on the input driving the dynamics, which do not match the reality of RNN applications. Trying to bridge this gap, we framed RNNs dynamics driven by generic input sequences in the context of nonautonomous dynamical system theory. This brought us to enquire deeply into a fundamental principle established for RNNs known as the echo state property (ESP). In particular, we argue that input-driven RNNs can be reliable computational models even without satisfying the classical ESP formulation. We prove a sort of input-driven fixed point theorem and exploit it to (i) demonstrate the existence and uniqueness of a global attracting solution for strongly (in amplitude) input-driven RNNs, (ii) deduce the existence of multiple responses for certain input signals which can be reliably exploited for computational purposes, and (iii) study the stability of attracting solutions w.r.t. input sequences. Finally, we highlight the active role of the input in determining qualitative changes in the RNN dynamics, e.g. the number of stable responses, in contrast to commonly known qualitative changes due to variations of model parameters

Boolean operations on 3D selective Nef complexes : data structure, algorithms, optimized implementation, experiments and applications

Nef polyhedra in d-dimensional space are the closure of half-spaces under boolean set operations. Consequently, they can represent non-manifold situations, open and closed sets, mixed-dimensional complexes, and they are closed under all boolean and topological operations, such as complement and boundary. The generality of Nef complexes is essential for some applications. In this thesis, we present a new data structure for the boundary representation of three-dimensional Nef polyhedra and efficient algorithms for boolean operations. We use exact arithmetic to avoid well known problems with floating-point arithmetic and handle all degeneracies. Furthermore, we present important optimizations for the algorithms, and evaluate this optimized implementation with extensive experiments. The experiments supplement the theoretical runtime analysisNef-Polyeder sind d-dimensionale Punktmengen, die durch eine endliche Anzahl boolescher Operationen über Halbräumen generiert werden. Sie sind abgeschlossen hinsichtlich boolescher und topologischer Operationen. Als Konsequenz daraus können sie nicht-mannigfaltige Situationen, offene und geschlossene Mengen und gemischt-dimensionale Komplexe darstellen. Die Allgemeinheit von Nef-Komplexen ist unentbehrlich für einige Anwendungen. In dieser Doktorarbeit stellen wir eine neue Datenstruktur vor, die eine Randdarstellung von dreidimensionalen Nef-polyedern und Algorithmen für boolesche Operationen realisiert. Wir benutzen exakte Arithmetik um die bekannten Probleme mit Gleitkommaarithmetik und Degeneriertheiten zu vermeiden. Außerdem präsentieren wir wichtige Optimierungen der Algorithmen und bewerten die optimierte Implementierung an Hand umfassender Experimente. Weitere Experimente belegen die theoretische Laufzeitanalyse und vergleichen unsere Implementation mit dem kommerziellen CAD kernel ACIS. ACIS is meistens bis zu sechs mal schneller, aber es gibt auch Beispiele bei denen ACIS scheitert. Nef-Polyeder können bei einer Vielzahl von Anwendungen eingesetzt werden. Wir präsentieren einfache Implementationen zweier Anwendungen - von der visuellen Hülle und von der Minkowski-Summe zwei abgeschlossener Nef-Polyeder

Model system approach to the study of UV induced DNA protein crosslink reactions

In this thesis the photoreaction leading from 5-benzyluracil (5BU) to the formation of a cyclized compound, the 5,6BU is studied using several ab initio molecular dynamics computational methods. The reaction is studied as model for the UV induced DNA protein crosslink reaction. The exploration of the reaction mechanism provides new insights in the construction of an efficient methodology to stabilize with UV puled lasers the transient interactions between DNA and close lying proteins in a biological environment

Algorithms for Imaging Atmospheric Cherenkov Telescopes

Imaging Atmospheric Cherenkov Telescopes (IACTs) are complex instruments for ground-based -ray astronomy and require sophisticated software for the handling of the measured data. In part one of this work, a modular and efficient software framework is presented that allows to run the complete chain from reading the raw data from the telescopes, over calibration, background reduction and reconstruction, to the sky maps. Several new methods and fast algorithms have been developed and are presented. Furthermore, it was found that the currently used file formats in IACT experiments are not optimal in terms of flexibility and I/O speed. Therefore, in part two a new file format was developed, which allows to store the camera and subsystem data in all its complexity. It offers fast lossy and lossless compression optimized for the high data rates of IACT experiments. Since many other scientific experiments also struggle with enormous data rates, the compression algorithm was further optimized and generalized, and is now able to efficiently compress the data of other experiments as well. Finally, for those who prefer to store their data as ASCII text, a fast I/O scheme is presented, including the necessary compression and conversion routines. Although the second part of this thesis is very technical, it might still be interesting for scientists designing an experiment with high data rates

Riemann surfaces with symmetry: algorithms and applications

Riemann surfaces frequently possess automorphisms which can be exploited to simplify calculations. However, existing computer software (Maple in particular) is designed for maximum generality and has not yet been extended to make use of available symmetries. In many calculations, the symmetries can be most easily used by choosing a speci c basis for 1( ,Z) under which the automorphism group acts neatly. This thesis describes a Maple library, designed to be used in conjunction with the existing algcurves, which allows such a choice to be made. In addition we create a visual tool to simplify the presentation of Riemann surfaces as sheeted covers of C and the creation of homology bases suitable for use in the Maple library. Two applications are considered for these techniques, rst Klein's curve and then Bring's. Both of these possess maximal symmetry groups for their genus, and this fact is exploited to obtain neat algebraic homology bases. In the Klein case the basis is novel; Bring's is derived from work in the hyperbolic setting by Riera. In both cases previous hyperbolic work and calculations are related to the algebraic results. Vectors of Riemann constants are calculated for both curves, again exploiting the symmetry. Finally this thesis moves back to simpler cases, and presents a general algorithm to convert results from general genus 2 curves into results based on a symmetric basis if one exists. This is applied to algebraic and numeric examples where we discover an elliptic surface covered in each case

Thermal Effects in Physics and Dynamics of Small Bodies of the Solar System

Thermal Effects in Physics and Dynamics of Small Bodies of the Solar System Abstract of the Ph.D. thesis \s David Capek It has been shown, that the thermal effects are very important in the dynamics of small Solar System bodies. A phenomenon which is known as the Yarkovsky effect is able to secularly change the semimajor axis of an orbit, while the YORP effect affects the rotation state of a body. The Yarkovsky effect and the YORP effect were previously calculated with many constraining assumptions like spherical shapes of asteroids, circular orbits, small variations of the surface temperature, principal axis rotation, constant thermal parameters, etc. We developed a sophisticated numerical model of the Yarkovsky/YORP effect without such simplifications. With this model, we have been able to describe the shape, the orbit, the rotation and the thermal behaviour of an asteroid in a very precise way. The YORP effect was studied on a sample of artificially generated shapes, roughly resembling Main Belt asteroids, and also on several shapes of real asteroids. The depen- dence of YORP on the obliquity and the thermal parameters of the surface were studied (Vokrouhlicky and Capek, 2002; Capek and Vokrouhlicky, 2004). A wide variety of pos- sible YORP evolution paths of the spin state was found. The possibility of...Tepelne jevy ve fyzice a dynamice malych teles slunecm soustavy Abstrakt dizertacnf prace David Capek Behem posledni doby se ukazalo, ze tepelne jevy jsou velmi vyznamne v dynamice malych teles slunecni soustavy. Intenzivne studovan byl predevslm jev zvany Jarkovskeho efekt, ktery je schopen dlouhodobe menit velkou poloosn drahy a YORP efekt, jez ovlivnuje rotacni stav telesa. Jarkovskeho a YORP efekt byly drive pocitany s mnoha omezujicimi predpoklady. Napnklad byly uvazovany kulove tvary asteroidu, kruhove drahy, male variace povrchove teploty, rotace okolo hlavni osy tenzoru setrvacnosti, konstantni tepelne parametry a podobne. Proto jsme vyvinnli numericky model pro vypocet Jarkovskeho/YORP jevu, ktery neni omezen temito predpoklady. S timto modelem jsme byli schopni velmi pfesne popsat tvar, drahu, rotaci a tepelne vlastnosti studovaneho telesa. YORP efekt byl studovan na vzorku umele vytvofenych tvaru} ktere odpovidaji aste- roidum hlavniho pasu, a take na tvarech skutecnych asteroidu. Zkoumali jsme zejmena savislost YORP jevu na obliquite a na tepelnych parametrech povrchu. Byla zjistena a diskutovana siroka skala moznosti v)'voje rotacniho stavu asteroidu (Vokrouhlickj'- and Capek, 2002; Capek and Vokrouhlickyj 2004). Pro nektere asteroidy bylo predpovezeno, ze Ize v budoucnosti ocekavat uspesnou...Institute of Theoretical PhysicsÚstav teoretické fyzikyFaculty of Mathematics and PhysicsMatematicko-fyzikální fakult

Extended Finite Element Methods for Brittle and Cohesive Fracture

The safety of engineering structures depends heavily on the presence of cracks, which may propagate and lead eventually to structural failure. This dissertation aims to advance the computational modeling of fracture, within the context of linear elastic fracture mechanics (LEFM) and cohesive zone models (CZMs). The extended finite element method (XFEM) is employed as the discretization method and cracks in both homogeneous and bimaterial solids are considered in this work. First, a novel set of enrichment functions within the framework of XFEM is proposed for the LEFM analysis of interface cracks in bimaterials. The motivation for the new enrichment set stems from the revelation that the accuracy of the widely accepted 12-fold bimaterial enrichment functions significantly deteriorates with the increase in material mismatch. To this end, we propose an 8-fold material-dependent enrichment set, derived from the analytical asymptotic displacement field, that well captures the near-tip oscillating singular fields of interface cracks, including the transition to weak discontinuities of bimaterials. The new enrichment set is tested on various examples and found to outperform the 12-fold set in terms of accuracy, conditioning, and total number of degrees of freedom (DOFs). The formulation is then extended to include high-order enrichment functions and accurate stress and displacement fields are obtained. The complex stress intensity factors (SIFs) of interface cracks are evaluated by employing Irwin's crack closure integral. To this end, a closed-form SIF formulation in terms of the enriched DOFs is derived by matching the leading term in the XFEM with an analytical expression of Irwin's integral. Hence, the SIFs of interface cracks can be directly obtained upon the solution of the XFEM discrete system without cumbersome post-processing requirements. The proposed method is shown to work well on several benchmark examples involving straight and curved interface cracks, giving accurate SIF results. Another contribution of the work is the application of Irwin's integral to the estimation of SIFs for curved homogeneous cracks. At the core, the proposed approach employs high-order enrichment functions to accurately capture the near-tip fields and evaluates the original definition of Irwin's integral through closed-form formulations in terms of enriched DOFs. An improved quadrature scheme using high-order isoparametric mapping together with a generalized Duffy transformation is proposed to integrate singular fields in tip elements with curved cracks. The proposed extraction approach is shown to yield decomposed SIFs with excellent accuracy and avoid the need for auxiliary fields as in J-integral method. Second, with respect to cohesive fracture, a discrete damage zone model (DDZM) is proposed following a rigorous thermodynamic framework similar to that of continuum damage mechanics (CDM). For the modeling of mixed-mode delamination, a novel damage evolution law is proposed to account for the coupled interaction between opening and sliding modes of interface deformations. A comprehensive comparison made with several popular CZMs in the literature demonstrates the thermodynamic consistency of the DDZM. The proposed interface model is integrated with the XFEM and the effectiveness of this framework has been validated on various benchmark problems. Finally, a novel continuous/discontinuous method is proposed to simulate the entire failure process of quasi-brittle materials: from the nucleation of diffuse damage to the development of discrete cracks . An integral-type nonlocal continuum damage model is coupled in this framework with DDZM with a new numerical energetic coupling scheme. The transition from the continuous (CDM) to the discontinuous approach (DDZM) can be triggered at any damage level with a weak energetic equivalence preserved. A few benchmark problems involving straight and curved cracks are investigated to demonstrate the applicability and robustness of the coupled XFEM cohesive-damage approach