70 research outputs found

    Topos and Stacks of Deep Neural Networks

    Full text link
    Every known artificial deep neural network (DNN) corresponds to an object in a canonical Grothendieck's topos; its learning dynamic corresponds to a flow of morphisms in this topos. Invariance structures in the layers (like CNNs or LSTMs) correspond to Giraud's stacks. This invariance is supposed to be responsible of the generalization property, that is extrapolation from learning data under constraints. The fibers represent pre-semantic categories (Culioli, Thom), over which artificial languages are defined, with internal logics, intuitionist, classical or linear (Girard). Semantic functioning of a network is its ability to express theories in such a language for answering questions in output about input data. Quantities and spaces of semantic information are defined by analogy with the homological interpretation of Shannon's entropy (P.Baudot and D.B. 2015). They generalize the measures found by Carnap and Bar-Hillel (1952). Amazingly, the above semantical structures are classified by geometric fibrant objects in a closed model category of Quillen, then they give rise to homotopical invariants of DNNs and of their semantic functioning. Intentional type theories (Martin-Loef) organize these objects and fibrations between them. Information contents and exchanges are analyzed by Grothendieck's derivators

    Categorial Modal Realism

    Get PDF
    The current conception of the plurality of worlds is founded on a set theoretic understanding of possibilia. This paper provides an alternative category theoretic conception and argues that it is at least as serviceable for our understanding of possibilia. In addition to or instead of the notion of possibilia conceived as possible objects or possible individuals, this alternative to set theoretic modal realism requires the notion of possible morphisms, conceived as possible changes, processes or transformations. To support this alternative conception of the plurality of worlds, I provide two examples where a category theoretic account can do work traditionally done by the set theoretic account: one on modal logic and another on paradoxes of size. I argue that the categorial account works at least as well as the set theoretic account, and moreover suggest that it has something to add in each case: it makes apparent avenues of inquiry that were obscured, if not invisible, on the set theoretic account. I conclude with a plea for epistemological humility about our acceptance of either a category-like or set-like realist ontology of modality

    Dynamical systems via domains:Toward a unified foundation of symbolic and non-symbolic computation

    Get PDF
    Non-symbolic computation (as, e.g., in biological and artificial neural networks) is astonishingly good at learning and processing noisy real-world data. However, it lacks the kind of understanding we have of symbolic computation (as, e.g., specified by programming languages). Just like symbolic computation, also non-symbolic computation needs a semantics—or behavior description—to achieve structural understanding. Domain theory has provided this for symbolic computation, and this thesis is about extending it to non-symbolic computation. Symbolic and non-symbolic computation can be described in a unified framework as state-discrete and state-continuous dynamical systems, respectively. So we need a semantics for dynamical systems: assigning to a dynamical system a domain—i.e., a certain mathematical structure—describing the system’s behavior. In part 1 of the thesis, we provide this domain-theoretic semantics for the ‘symbolic’ state-discrete systems (i.e., labeled transition systems). And in part 2, we do this for the ‘non-symbolic’ state-continuous systems (known from ergodic theory). This is a proper semantics in that the constructions form functors (in the sense of category theory) and, once appropriately formulated, even adjunctions and, stronger yet, equivalences. In part 3, we explore how this semantics relates the two types of computation. It suggests that non-symbolic computation is the limit of symbolic computation (in the ‘profinite’ sense). Conversely, if the system’s behavior is fairly stable, it may be described as realizing symbolic computation (here the concepts of ergodicity and algorithmic randomness are useful). However, the underlying concept of stability is limited by a no-go result due to a novel interpretation of Fitch’s paradox. This also has implications for AI-safety and, more generally, suggests fruitful applications of philosophical tools in the non-symbolic computation of modern AI
    • 

    corecore