Empiricism without Magic: Transformational Abstraction in Deep Convolutional Neural Networks

In artificial intelligence, recent research has demonstrated the remarkable potential of Deep Convolutional Neural Networks (DCNNs), which seem to exceed state-of-the-art performance in new domains weekly, especially on the sorts of very difficult perceptual discrimination tasks that skeptics thought would remain beyond the reach of artificial intelligence. However, it has proven difficult to explain why DCNNs perform so well. In philosophy of mind, empiricists have long suggested that complex cognition is based on information derived from sensory experience, often appealing to a faculty of abstraction. Rationalists have frequently complained, however, that empiricists never adequately explained how this faculty of abstraction actually works. In this paper, I tie these two questions together, to the mutual benefit of both disciplines. I argue that the architectural features that distinguish DCNNs from earlier neural networks allow them to implement a form of hierarchical processing that I call “transformational abstraction”. Transformational abstraction iteratively converts sensory-based representations of category exemplars into new formats that are increasingly tolerant to “nuisance variation” in input. Reflecting upon the way that DCNNs leverage a combination of linear and non-linear processing to efficiently accomplish this feat allows us to understand how the brain is capable of bi-directional travel between exemplars and abstractions, addressing longstanding problems in empiricist philosophy of mind. I end by considering the prospects for future research on DCNNs, arguing that rather than simply implementing 80s connectionism with more brute-force computation, transformational abstraction counts as a qualitatively distinct form of processing ripe with philosophical and psychological significance, because it is significantly better suited to depict the generic mechanism responsible for this important kind of psychological processing in the brain

A parallel multistate framework for atomistic non-equilibrium reaction dynamics of solutes in strongly interacting organic solvents

We describe a parallel linear-scaling computational framework developed to implement arbitrarily large multi-state empirical valence bond (MS-EVB) calculations within CHARMM. Forces are obtained using the Hellman-Feynmann relationship, giving continuous gradients, and excellent energy conservation. Utilizing multi-dimensional Gaussian coupling elements fit to CCSD(T)-F12 electronic structure theory, we built a 64-state MS-EVB model designed to study the F + CD3CN -> DF + CD2CN reaction in CD3CN solvent. This approach allows us to build a reactive potential energy surface (PES) whose balanced accuracy and efficiency considerably surpass what we could achieve otherwise. We use our PES to run MD simulations, and examine a range of transient observables which follow in the wake of reaction, including transient spectra of the DF vibrational band, time dependent profiles of vibrationally excited DF in CD3CN solvent, and relaxation rates for energy flow from DF into the solvent, all of which agree well with experimental observations. Immediately following deuterium abstraction, the nascent DF is in a non-equilibrium regime in two different respects: (1) it is highly excited, with ~23 kcal mol-1 localized in the stretch; and (2) not yet Hydrogen bonded to the CD3CN solvent, its microsolvation environment is intermediate between the non-interacting gas-phase limit and the solution-phase equilibrium limit. Vibrational relaxation of the nascent DF results in a spectral blue shift, while relaxation of its microsolvation environment results in a red shift. These two competing effects result in a post-reaction relaxation profile distinct from that observed when DF vibration excitation occurs within an equilibrium microsolvation environment. The parallel software framework presented in this paper should be more broadly applicable to a range of complex reactive systems.Comment: 58 pages and 29 Figure

Usage-based and emergentist approaches to language acquisition

It was long considered to be impossible to learn grammar based on linguistic experience alone. In the past decade, however, advances in usage-based linguistic theory, computational linguistics, and developmental psychology changed the view on this matter. So-called usage-based and emergentist approaches to language acquisition state that language can be learned from language use itself, by means of social skills like joint attention, and by means of powerful generalization mechanisms. This paper first summarizes the assumptions regarding the nature of linguistic representations and processing. Usage-based theories are nonmodular and nonreductionist, i.e., they emphasize the form-function relationships, and deal with all of language, not just selected levels of representations. Furthermore, storage and processing is considered to be analytic as well as holistic, such that there is a continuum between children's unanalyzed chunks and abstract units found in adult language. In the second part, the empirical evidence is reviewed. Children's linguistic competence is shown to be limited initially, and it is demonstrated how children can generalize knowledge based on direct and indirect positive evidence. It is argued that with these general learning mechanisms, the usage-based paradigm can be extended to multilingual language situations and to language acquisition under special circumstances

Enzymatic functionalization of carbon-hydrogen bonds

The development of new catalytic methods to functionalize carbon–hydrogen (C–H) bonds continues to progress at a rapid pace due to the significant economic and environmental benefits of these transformations over traditional synthetic methods. In nature, enzymes catalyze regio- and stereoselective C–H bond functionalization using transformations ranging from hydroxylation to hydroalkylation under ambient reaction conditions. The efficiency of these enzymes relative to analogous chemical processes has led to their increased use as biocatalysts in preparative and industrial applications. Furthermore, unlike small molecule catalysts, enzymes can be systematically optimized via directed evolution for a particular application and can be expressed in vivo to augment the biosynthetic capability of living organisms. While a variety of technical challenges must still be overcome for practical application of many enzymes for C–H bond functionalization, continued research on natural enzymes and on novel artificial metalloenzymes will lead to improved synthetic processes for efficient synthesis of complex molecules. In this critical review, we discuss the most prevalent mechanistic strategies used by enzymes to functionalize non-acidic C–H bonds, the application and evolution of these enzymes for chemical synthesis, and a number of potential biosynthetic capabilities uniquely enabled by these powerful catalysts (110 references)

Challenging the Computational Metaphor: Implications for How We Think

This paper explores the role of the traditional computational metaphor in our thinking as computer scientists, its influence on epistemological styles, and its implications for our understanding of cognition. It proposes to replace the conventional metaphor--a sequence of steps--with the notion of a community of interacting entities, and examines the ramifications of such a shift on these various ways in which we think