Integrating Symbolic and Neural Processing in a Self-Organizing Architechture for Pattern Recognition and Prediction

British Petroleum (89A-1204); Defense Advanced Research Projects Agency (N00014-92-J-4015); National Science Foundation (IRI-90-00530); Office of Naval Research (N00014-91-J-4100); Air Force Office of Scientific Research (F49620-92-J-0225

Constraining Implicit Space with Minimum Description Length: An Unsupervised Attention Mechanism across Neural Network Layers

Inspired by the adaptation phenomenon of neuronal firing, we propose the regularity normalization (RN) as an unsupervised attention mechanism (UAM) which computes the statistical regularity in the implicit space of neural networks under the Minimum Description Length (MDL) principle. Treating the neural network optimization process as a partially observable model selection problem, UAM constrains the implicit space by a normalization factor, the universal code length. We compute this universal code incrementally across neural network layers and demonstrated the flexibility to include data priors such as top-down attention and other oracle information. Empirically, our approach outperforms existing normalization methods in tackling limited, imbalanced and non-stationary input distribution in image classification, classic control, procedurally-generated reinforcement learning, generative modeling, handwriting generation and question answering tasks with various neural network architectures. Lastly, UAM tracks dependency and critical learning stages across layers and recurrent time steps of deep networks

Adaptive Resonance Theory: Self-Organizing Networks for Stable Learning, Recognition, and Prediction

Adaptive Resonance Theory (ART) is a neural theory of human and primate information processing and of adaptive pattern recognition and prediction for technology. Biological applications to attentive learning of visual recognition categories by inferotemporal cortex and hippocampal system, medial temporal amnesia, corticogeniculate synchronization, auditory streaming, speech recognition, and eye movement control are noted. ARTMAP systems for technology integrate neural networks, fuzzy logic, and expert production systems to carry out both unsupervised and supervised learning. Fast and slow learning are both stable response to large non stationary databases. Match tracking search conjointly maximizes learned compression while minimizing predictive error. Spatial and temporal evidence accumulation improve accuracy in 3-D object recognition. Other applications are noted.Office of Naval Research (N00014-95-I-0657, N00014-95-1-0409, N00014-92-J-1309, N00014-92-J4015); National Science Foundation (IRI-94-1659

Catching up with Method and Process Practice: An Industry-Informed Baseline for Researchers

Software development methods are usually not applied by the book.companies are under pressure to continuously deploy software products that meet market needs and stakeholders\u27 requests. To implement efficient and effective development processes, companies utilize multiple frameworks, methods and practices, and combine these into hybrid methods. A common combination contains a rich management framework to organize and steer projects complemented with a number of smaller practices providing the development teams with tools to complete their tasks. In this paper, based on 732 data points collected through an international survey, we study the software development process use in practice. Our results show that 76.8% of the companies implement hybrid methods.company size as well as the strategy in devising and evolving hybrid methods affect the suitability of the chosen process to reach company or project goals. Our findings show that companies that combine planned improvement programs with process evolution can increase their process\u27 suitability by up to 5%

A Self-Organizing Neural System for Learning to Recognize Textured Scenes

A self-organizing ARTEX model is developed to categorize and classify textured image regions. ARTEX specializes the FACADE model of how the visual cortex sees, and the ART model of how temporal and prefrontal cortices interact with the hippocampal system to learn visual recognition categories and their names. FACADE processing generates a vector of boundary and surface properties, notably texture and brightness properties, by utilizing multi-scale filtering, competition, and diffusive filling-in. Its context-sensitive local measures of textured scenes can be used to recognize scenic properties that gradually change across space, as well a.s abrupt texture boundaries. ART incrementally learns recognition categories that classify FACADE output vectors, class names of these categories, and their probabilities. Top-down expectations within ART encode learned prototypes that pay attention to expected visual features. When novel visual information creates a poor match with the best existing category prototype, a memory search selects a new category with which classify the novel data. ARTEX is compared with psychophysical data, and is benchmarked on classification of natural textures and synthetic aperture radar images. It outperforms state-of-the-art systems that use rule-based, backpropagation, and K-nearest neighbor classifiers.Defense Advanced Research Projects Agency; Office of Naval Research (N00014-95-1-0409, N00014-95-1-0657

Hidden innovation in the construction and property sectors

The construction and property industries have a poor reputation for innovation. Indeed, this reputation appears to be backed by official statistics which projects these industries as being devoid of innovative activity. However, the reputation is undeserved. Official statistics misrepresent the extent and nature of innovation in these sectors. A closer analysis reveals that much of the innovation that exists in the sector is hidden from conventional measures. By its nature therefore, it is difficult to measure the precise extent of this innovation and disaggregate it from general improvement. This opaqueness strengthens the need for policymakers, practitioners and researchers within, amongst others, the surveying sector to go beyond the visible spectrum of innovation and design and implement appropriate policies, knowledge bases and practices which engage and leverage the hitherto hidden aspects of innovation

Thriving in the 21st century: Learning Literacies for the Digital Age (LLiDA project): Executive Summary, Conclusions and Recommendations

LLiDA set out to: review the evidence of change in the contexts of learning, including the nature of work,nknowledge, social life and citizenship, communications media and other technologies review current responses to these challenges from the further and higher education sectors, in terms of: a) the kinds of capabilities valued, taught for and assessed (especially as revealed through competence frameworks); b) the ways in which capabilities are supported ('provision') c) the value placed on staff and student 'literacies of the digital' collect original data concerning current practice in literacies provision in UK FE and HE, including 15 institutional audits and over 40 examples of forward thinking practice offer conclusions and recommendations, in terms of the same issues reviewed in