Formal Modeling of Connectionism using Concurrency Theory, an Approach Based on Automata and Model Checking

This paper illustrates a framework for applying formal methods techniques, which are symbolic in nature, to specifying and verifying neural networks, which are sub-symbolic in nature. The paper describes a communicating automata [Bowman & Gomez, 2006] model of neural networks. We also implement the model using timed automata [Alur & Dill, 1994] and then undertake a verification of these models using the model checker Uppaal [Pettersson, 2000] in order to evaluate the performance of learning algorithms. This paper also presents discussion of a number of broad issues concerning cognitive neuroscience and the debate as to whether symbolic processing or connectionism is a suitable representation of cognitive systems. Additionally, the issue of integrating symbolic techniques, such as formal methods, with complex neural networks is discussed. We then argue that symbolic verifications may give theoretically well-founded ways to evaluate and justify neural learning systems in the field of both theoretical research and real world applications

Phototaxic foraging of the archaepaddler, a hypothetical deep-sea species

An autonomous agent (animat, hypothetical animal), called the (archae) paddler, is simulated in sufficient detail to regard its simulated aquatic locomotion (paddling) as physically possible. The paddler is supposed to be a model of an animal that might exist, although it is perfectly possible to view it as a model of a robot that might be built. The agent is assumed to navigate in a simulated deep-sea environment, where it hunts autoluminescent prey. It uses a biologically inspired phototaxic foraging-strategy, while paddling in a layer just above the bottom. The advantage of this living space is that the navigation problem is essentially two-dimensional. Moreover, the deep-sea environment is physically simple (and hence easier to simulate): no significant currents, constant temperature, completely dark. A foraging performance metric is developed that circumvents the necessity to solve the travelling salesman problem. A parametric simulation study then quantifies the influence of habitat factors, such as the density of prey, and the body-geometry (e.g. placement, direction and directional selectivity of the eyes) on foraging success. Adequate performance proves to require a specific body-% geometry adapted to the habitat characteristics. In general performance degrades smoothly for modest changes of the geometric and habitat parameters, indicating that we work in a stable region of 'design space'. The parameters have to strike a compromise between on the one hand the ability to 'fixate' an attractive target, and on the other hand to 'see' as many targets at the same time as possible. One important conclusion is that simple reflex-based navigation can be surprisingly efficient. In the second place, performance in a global task (foraging) depends strongly on local parameters like visual direction-tuning, position of the eyes and paddles, etc. Behaviour and habitat 'mould' the body, and the body-geometry strongly influences performance. The resulting platform enables further testing of foraging strategies, or vision and locomotion theories stemming either from biology or from robotics

Empiricism without Magic: Transformational Abstraction in Deep Convolutional Neural Networks

In artificial intelligence, recent research has demonstrated the remarkable potential of Deep Convolutional Neural Networks (DCNNs), which seem to exceed state-of-the-art performance in new domains weekly, especially on the sorts of very difficult perceptual discrimination tasks that skeptics thought would remain beyond the reach of artificial intelligence. However, it has proven difficult to explain why DCNNs perform so well. In philosophy of mind, empiricists have long suggested that complex cognition is based on information derived from sensory experience, often appealing to a faculty of abstraction. Rationalists have frequently complained, however, that empiricists never adequately explained how this faculty of abstraction actually works. In this paper, I tie these two questions together, to the mutual benefit of both disciplines. I argue that the architectural features that distinguish DCNNs from earlier neural networks allow them to implement a form of hierarchical processing that I call “transformational abstraction”. Transformational abstraction iteratively converts sensory-based representations of category exemplars into new formats that are increasingly tolerant to “nuisance variation” in input. Reflecting upon the way that DCNNs leverage a combination of linear and non-linear processing to efficiently accomplish this feat allows us to understand how the brain is capable of bi-directional travel between exemplars and abstractions, addressing longstanding problems in empiricist philosophy of mind. I end by considering the prospects for future research on DCNNs, arguing that rather than simply implementing 80s connectionism with more brute-force computation, transformational abstraction counts as a qualitatively distinct form of processing ripe with philosophical and psychological significance, because it is significantly better suited to depict the generic mechanism responsible for this important kind of psychological processing in the brain

Deep Shading: Convolutional Neural Networks for Screen-Space Shading

In computer vision, Convolutional Neural Networks (CNNs) have recently achieved new levels of performance for several inverse problems where RGB pixel appearance is mapped to attributes such as positions, normals or reflectance. In computer graphics, screen-space shading has recently increased the visual quality in interactive image synthesis, where per-pixel attributes such as positions, normals or reflectance of a virtual 3D scene are converted into RGB pixel appearance, enabling effects like ambient occlusion, indirect light, scattering, depth-of-field, motion blur, or anti-aliasing. In this paper we consider the diagonal problem: synthesizing appearance from given per-pixel attributes using a CNN. The resulting Deep Shading simulates all screen-space effects as well as arbitrary combinations thereof at competitive quality and speed while not being programmed by human experts but learned from example images