The evolution of a visual-to-auditory sensory substitution device using interactive genetic algorithms

Sensory Substitution is a promising technique for mitigating the loss of a sensory modality. Sensory Substitution Devices (SSDs) work by converting information from the impaired sense (e.g. vision) into another, intact sense (e.g. audition). However, there are a potentially infinite number of ways of converting images into sounds and it is important that the conversion takes into account the limits of human perception and other user-related factors (e.g. whether the sounds are pleasant to listen to). The device explored here is termed “polyglot” because it generates a very large set of solutions. Specifically, we adapt a procedure that has been in widespread use in the design of technology but has rarely been used as a tool to explore perception – namely Interactive Genetic Algorithms. In this procedure, a very large range of potential sensory substitution devices can be explored by creating a set of ‘genes’ with different allelic variants (e.g. different ways of translating luminance into loudness). The most successful devices are then ‘bred’ together and we statistically explore the characteristics of the selected-for traits after multiple generations. The aim of the present study is to produce design guidelines for a better SSD. In three experiments we vary the way that the fitness of the device is computed: by asking the user to rate the auditory aesthetics of different devices (Experiment 1), by measuring the ability of participants to match sounds to images (Experiment 2) and the ability to perceptually discriminate between two sounds derived from similar images (Experiment 3). In each case the traits selected for by the genetic algorithm represent the ideal SSD for that task. Taken together, these traits can guide the design of a better SSD

Conditional Random Field Autoencoders for Unsupervised Structured Prediction

We introduce a framework for unsupervised learning of structured predictors with overlapping, global features. Each input's latent representation is predicted conditional on the observable data using a feature-rich conditional random field. Then a reconstruction of the input is (re)generated, conditional on the latent structure, using models for which maximum likelihood estimation has a closed-form. Our autoencoder formulation enables efficient learning without making unrealistic independence assumptions or restricting the kinds of features that can be used. We illustrate insightful connections to traditional autoencoders, posterior regularization and multi-view learning. We show competitive results with instantiations of the model for two canonical NLP tasks: part-of-speech induction and bitext word alignment, and show that training our model can be substantially more efficient than comparable feature-rich baselines

Chance and Necessity in Evolution: Lessons from RNA

The relationship between sequences and secondary structures or shapes in RNA exhibits robust statistical properties summarized by three notions: (1) the notion of a typical shape (that among all sequences of fixed length certain shapes are realized much more frequently than others), (2) the notion of shape space covering (that all typical shapes are realized in a small neighborhood of any random sequence), and (3) the notion of a neutral network (that sequences folding into the same typical shape form networks that percolate through sequence space). Neutral networks loosen the requirements on the mutation rate for selection to remain effective. The original (genotypic) error threshold has to be reformulated in terms of a phenotypic error threshold. With regard to adaptation, neutrality has two seemingly contradictory effects: It acts as a buffer against mutations ensuring that a phenotype is preserved. Yet it is deeply enabling, because it permits evolutionary change to occur by allowing the sequence context to vary silently until a single point mutation can become phenotypically consequential. Neutrality also influences predictability of adaptive trajectories in seemingly contradictory ways. On the one hand it increases the uncertainty of their genotypic trace. At the same time neutrality structures the access from one shape to another, thereby inducing a topology among RNA shapes which permits a distinction between continuous and discontinuous shape transformations. To the extent that adaptive trajectories must undergo such transformations, their phenotypic trace becomes more predictable.Comment: 37 pages, 14 figures; 1998 CNLS conference; high quality figures at http://www.santafe.edu/~walte

Categorical invariance and structural complexity in human concept learning

An alternative account of human concept learning based on an invariance measure of the categorical\ud stimulus is proposed. The categorical invariance model (CIM) characterizes the degree of structural\ud complexity of a Boolean category as a function of its inherent degree of invariance and its cardinality or\ud size. To do this we introduce a mathematical framework based on the notion of a Boolean differential\ud operator on Boolean categories that generates the degrees of invariance (i.e., logical manifold) of the\ud category in respect to its dimensions. Using this framework, we propose that the structural complexity\ud of a Boolean category is indirectly proportional to its degree of categorical invariance and directly\ud proportional to its cardinality or size. Consequently, complexity and invariance notions are formally\ud unified to account for concept learning difficulty. Beyond developing the above unifying mathematical\ud framework, the CIM is significant in that: (1) it precisely predicts the key learning difficulty ordering of\ud the SHJ [Shepard, R. N., Hovland, C. L.,&Jenkins, H. M. (1961). Learning and memorization of classifications.\ud Psychological Monographs: General and Applied, 75(13), 1-42] Boolean category types consisting of three\ud binary dimensions and four positive examples; (2) it is, in general, a good quantitative predictor of the\ud degree of learning difficulty of a large class of categories (in particular, the 41 category types studied\ud by Feldman [Feldman, J. (2000). Minimization of Boolean complexity in human concept learning. Nature,\ud 407, 630-633]); (3) it is, in general, a good quantitative predictor of parity effects for this large class of\ud categories; (4) it does all of the above without free parameters; and (5) it is cognitively plausible (e.g.,\ud cognitively tractable)

Low-frequency oscillatory correlates of auditory predictive processing in cortical-subcortical networks: a MEG-study

Emerging evidence supports the role of neural oscillations as a mechanism for predictive information processing across large-scale networks. However, the oscillatory signatures underlying auditory mismatch detection and information flow between brain regions remain unclear. To address this issue, we examined the contribution of oscillatory activity at theta/alpha-bands (4–8/8–13 Hz) and assessed directed connectivity in magnetoencephalographic data while 17 human participants were presented with sound sequences containing predictable repetitions and order manipulations that elicited prediction-error responses. We characterized the spectro-temporal properties of neural generators using a minimum-norm approach and assessed directed connectivity using Granger Causality analysis. Mismatching sequences elicited increased theta power and phase-locking in auditory, hippocampal and prefrontal cortices, suggesting that theta-band oscillations underlie prediction-error generation in cortical-subcortical networks. Furthermore, enhanced feedforward theta/alpha-band connectivity was observed in auditory-prefrontal networks during mismatching sequences, while increased feedback connectivity in the alpha-band was observed between hippocampus and auditory regions during predictable sounds. Our findings highlight the involvement of hippocampal theta/alpha-band oscillations towards auditory prediction-error generation and suggest a spectral dissociation between inter-areal feedforward vs. feedback signalling, thus providing novel insights into the oscillatory mechanisms underlying auditory predictive processing