Prestructuring Neural Networks via Extended Dependency Analysis with Application to Pattern Classification

We consider the problem of matching domain-specific statistical structure to neural-network (NN) architecture. In past work we have considered this problem in the function approximation context; here we consider the pattern classification context. General Systems Methodology tools for finding problem-domain structure suffer exponential scaling of computation with respect to the number of variables considered. Therefore we introduce the use of Extended Dependency Analysis (EDA), which scales only polynomially in the number of variables, for the desired analysis. Based on EDA, we demonstrate a number of NN pre-structuring techniques applicable for building neural classifiers. An example is provided in which EDA results in significant dimension reduction of the input space, as well as capability for direct design of an NN classifier

Higher-level Application of Adaptive Dynamic Programming/reinforcement Learning – A Next phase for Controls and System Identification?

Humans have the ability to make use of experience while performing system identification and selecting control actions for changing situations. In contrast to current technological implementations that slow down as more knowledge is stored, as more experience is gained, human processing speeds up and has enhanced effectiveness. An emerging experience-based (“higher level”) approach promises to endow our technology with enhanced efficiency and effectiveness. The notions of context and context discernment are important to understanding this human ability. These are defined as appropriate to controls and system-identification. Some general background on controls, Dynamic Programming, and Adaptive Critic leading to Adaptive Dynamic Programming (ADP) will be provided. The higher-level application of Adaptive Dynamic Programming (ADP) is described, wherein ADP is employed to develop on-line algorithms that respond to changes in context by efficiently and effectively selecting designs from a repository of existing controller solutions– in contrast to the usual application of ADP that focuses on designing controllers directly. In this way, the ADP is said to be applied up a level from typical application. Key components of the approach include the notions of context, context discernment, and experience. These apply to applications in control and also to system identification. Details of the approach and its rationale will be described, including examples and recent developments of the underlying ideas.https://pdxscholar.library.pdx.edu/systems_science_seminar_series/1054/thumbnail.jp

Adaptive Dynamic Programming Approach to Experience-Based Systems Identification and Control

Humans have the ability to make use of experience while selecting their control actions for distinct and changing situations, and their process speeds up and have enhanced effectiveness as more experience is gained. In contrast, current technological implementations slow down as more knowledge is stored. A novel way of employing Approximate (or Adaptive) Dynamic Programming (ADP) is described that shifts the underlying Adaptive Critic type of Reinforcement Learning method up a level , away from designing individual (optimal) controllers to that of developing on-line algorithms that efficiently and effectively select designs from a repository of existing controller solutions (perhaps previously developed via application of ADP methods). The resulting approach is called Higher-Level Learning Algorithm. The approach and its rationale are described and some examples of its application are given. The notions of context and context discernment are important to understanding the human abilities noted above. These are first defined, in a manner appropriate to controls and system-identification, and as a foundation relating to the application arena, a historical view of the various phases during development of the controls field is given, organized by how the notion \u27context\u27 was, or was not, involved in each phase

Conceptual Graph Knowledge Systems as Problem Context for Neural Networks

For a connectionist network to be able to learn to generalize well, there must be some correspondence between the structure/constraints of the net\u27s architecture and those of the given problem space. Therefore, recourse to experiments with real-world problems will always be required in connectionist research. The author gives a outline of a problem area for which connectionist nets hold great promise: knowledge systems, where the knowledge is encoded/represented using conceptual graphs. Certain aspects of this problem context are already known, and these are probed for possible implementation by connectionist nets. The approach used is to present some basic properties of conceptual graphs, indicate operations important in their application, and point out those that might be candidates for implementation with neural nets. A special representation schema for conceptual graphs is used for their implementation by neural nets

Higher Level Application of ADP: A Next Phase for the Control Field?

Two distinguishing features of humanlike control vis-a-vis current technological control are the ability to make use of experience while selecting a control policy for distinct situations and the ability to do so faster and faster as more experience is gained (in contrast to current technological implementations that slow down as more knowledge is stored). The notions of context and context discernment are important to understanding this human ability. Whereas methods known as adaptive control and learning control focus on modifying the design of a controller as changes in context occur, experience-based (EB) control entails selecting a previously designed controller that is appropriate to the current situation. Developing the EB approach entails a shift of the technologist\u27s focus ldquoup a levelrdquo away from designing individual (optimal) controllers to that of developing online algorithms that efficiently and effectively select designs from a repository of existing controller solutions. A key component of the notions presented here is that of higher level learning algorithm. This is a new application of reinforcement learning and, in particular, approximate dynamic programming, with its focus shifted to the posited higher level, and is employed, with very promising results. The author\u27s hope for this paper is to inspire and guide future work in this promising area

On the Human Aspects in Structural Modeling

This article deals with two facets of structural modeling often ignored: a) the several human roles that effective participative modeling must encompass, and b) the group procedures developed to assist modelers in defining the elements of the system to be modeled. The roles are 1) the method technician, 2) the facilitator, and 3) the participant. Their recognition on the part of the technology assessment leader is vital to the successful conduct of participative modeling. The element-generating group procedures surveyed here are organized into two categories: those which emphasize an atmosphere for free-wheeling thinking, and those which emphasize structured guidance (either through use of words or geometric and analytic techniques). © 1979

On Systemness and the Problem Solver: Tutorial Comments

The main topic is the perceiving role played by the systems practitioner (SP) in the act of defining the system which is the subject of his problem-solving activities. A definition of system is given that makes explicit what perceptions must be made in Order for one to assert that an object under observation is a system. The corollary ideas of properties-of the-whole, suprasystem/system/subsystem, context, and (two-dimensional) multiple perspectives are discussed. The key advice is for the SP, during problem solving activity, to consciously adopt perceiving roles (perspectives), separately, at the supra-system, system, and sub-system levels, and to recognize that there is no such thing as an independent, objective perception of a system. The importance of the supra-system perspective to SP is improved ability to define context. The importance of multiple perspectives to SP is improved ability to consciously gather and organize a broad array of data about the problem. All these ideas are applicable to team problem solving. Copyright © 1986 by The Institute of Electrical and Electronics Engineers, Inc

On Comparing Neural Net Training Paradigms via Graded Pattern Recognition Tasks

A number of training paradigms for neural nets are under investigation by various researchers around the world. Usually, the efforts (as reported in the literature) focus on one of the paradigms, each contributing to the array of results being accumulated. The project to be reported in this paper, on the other hand, focuses on developing comparative information about a number of the paradigms. The training tasks for the networks are based on a set of pattern recognition problems. The data being used was created some 14 years ago while the author was at NASA developing machine implementable pattern recognition algorithms using then current (non neural-network) methodologies. The base data was in the form of aerial photographic imagery, and the task was to classify the images into one of five land use categories

Fuzziness and Catastrophe

In a recent short note, Flondor has alluded to a possible linkage of fuzzy set theory and catastrophe theory. We consider several features of catastrophe theory, namely the properties of discontinuous jumps, hysteresis, and divergence in the cusp catastrophe, and the role of the bias factor in the butterfly catastrophe, which have affinities to and suggest possible extensions of fuzzy set ideas. Certain functions extensively considered in catastrophe theory lend themselves in some cases to interpretation as membership functions. The use of such functions may be of interest for the characterization of linguistic descriptions which are time-varying and encompass both discrete and fuzzy distinctions