Generation of Explicit Knowledge from Empirical Data through Pruning of Trainable Neural Networks

This paper presents a generalized technology of extraction of explicit knowledge from data. The main ideas are 1) maximal reduction of network complexity (not only removal of neurons or synapses, but removal all the unnecessary elements and signals and reduction of the complexity of elements), 2) using of adjustable and flexible pruning process (the pruning sequence shouldn't be predetermined - the user should have a possibility to prune network on his own way in order to achieve a desired network structure for the purpose of extraction of rules of desired type and form), and 3) extraction of rules not in predetermined but any desired form. Some considerations and notes about network architecture and training process and applicability of currently developed pruning techniques and rule extraction algorithms are discussed. This technology, being developed by us for more than 10 years, allowed us to create dozens of knowledge-based expert systems. In this paper we present a generalized three-step technology of extraction of explicit knowledge from empirical data.Comment: 9 pages, The talk was given at the IJCNN '99 (Washington DC, July 1999

Correlations, Risk and Crisis: From Physiology to Finance

We study the dynamics of correlation and variance in systems under the load of environmental factors. A universal effect in ensembles of similar systems under the load of similar factors is described: in crisis, typically, even before obvious symptoms of crisis appear, correlation increases, and, at the same time, variance (and volatility) increases too. This effect is supported by many experiments and observations of groups of humans, mice, trees, grassy plants, and on financial time series. A general approach to the explanation of the effect through dynamics of individual adaptation of similar non-interactive individuals to a similar system of external factors is developed. Qualitatively, this approach follows Selye's idea about adaptation energy.Comment: 42 pages, 15 figures, misprints corrections, a proof is added, improved journal versio

Geometrical complexity of data approximators

There are many methods developed to approximate a cloud of vectors embedded in high-dimensional space by simpler objects: starting from principal points and linear manifolds to self-organizing maps, neural gas, elastic maps, various types of principal curves and principal trees, and so on. For each type of approximators the measure of the approximator complexity was developed too. These measures are necessary to find the balance between accuracy and complexity and to define the optimal approximations of a given type. We propose a measure of complexity (geometrical complexity) which is applicable to approximators of several types and which allows comparing data approximations of different types.Comment: 10 pages, 3 figures, minor correction and extensio

Evolution of adaptation mechanisms: adaptation energy, stress, and oscillating death

In 1938, H. Selye proposed the notion of adaptation energy and published "Experimental evidence supporting the conception of adaptation energy". Adaptation of an animal to different factors appears as the spending of one resource. Adaptation energy is a hypothetical extensive quantity spent for adaptation. This term causes much debate when one takes it literally, as a physical quantity, i.e. a sort of energy. The controversial points of view impede the systematic use of the notion of adaptation energy despite experimental evidence. Nevertheless, the response to many harmful factors often has general non-specific form and we suggest that the mechanisms of physiological adaptation admit a very general and nonspecific description. We aim to demonstrate that Selye's adaptation energy is the cornerstone of the top-down approach to modelling of non-specific adaptation processes. We analyse Selye's axioms of adaptation energy together with Goldstone's modifications and propose a series of models for interpretation of these axioms. {\em Adaptation energy is considered as an internal coordinate on the `dominant path' in the model of adaptation}. The phenomena of `oscillating death' and `oscillating remission' are predicted on the base of the dynamical models of adaptation. Natural selection plays a key role in the evolution of mechanisms of physiological adaptation. We use the fitness optimization approach to study of the distribution of resources for neutralization of harmful factors, during adaptation to a multifactor environment, and analyse the optimal strategies for different systems of factors

Reciprocal Relations Between Kinetic Curves

We study coupled irreversible processes. For linear or linearized kinetics with microreversibility, $\dot{x}=Kx$, the kinetic operator $K$ is symmetric in the entropic inner product. This form of Onsager's reciprocal relations implies that the shift in time, $\exp (Kt)$, is also a symmetric operator. This generates the reciprocity relations between the kinetic curves. For example, for the Master equation, if we start the process from the $i$th pure state and measure the probability $p_j(t)$ of the $j$th state ($j\neq i$), and, similarly, measure $p_i(t)$ for the process, which starts at the $j$th pure state, then the ratio of these two probabilities $p_j(t)/p_i(t)$ is constant in time and coincides with the ratio of the equilibrium probabilities. We study similar and more general reciprocal relations between the kinetic curves. The experimental evidence provided as an example is from the reversible water gas shift reaction over iron oxide catalyst. The experimental data are obtained using Temporal Analysis of Products (TAP) pulse-response studies. These offer excellent confirmation within the experimental error.Comment: 6 pages, 1 figure, the final versio

Influence of the thermal factor on the composition of electron-beam high-entropy ALTiVCrNbMo coatings

This paper reports results of studying the element and phase compositions of electron-beam coatings based on the high-entropy alloy AlTiVCrNbMo, depending on the deposition temperature (in the range of 300...700 °С). The high-entropy alloys were melted in an arc furnace in an atmosphere of high-purity argon. Vacuum condensates of the high-entropy alloy (AlTiVCrNbMo) with a thickness of 3–5 µm were obtained in the vacuum setup UVN-2M-1 at a working vacuum of 5·10-5 mТоrr. The alloy evaporation was performed from the water-cooled ingot mold using an electron-beam gun with a power of 5 kW. Condensation of vapors of all the elements of the alloy was performed onto copper substrates at temperatures of 300, 500, 700 °C. Based on analysis of the element composition of materials of the target made of the high-entropy six-element alloy AlTiVCrNbMo and electron-beam coatings, based on it, we established the critical parameter (specific heat of vaporization of an element) that defined a selective change in the element composition. In accordance with a characteristic change in the composition of coatings of the multi-element high-entropy alloy, 3 groups of elements were distinguished: with a specific heat of evaporation of 280...350 kJ/mol (group 1), 420…460 kJ/mol (group 2), and 590…680 kJ/mol (group 3). It was shown that the formation of a single-phase coating of the high-entropy alloy (based on BCC of the crystalline lattice) occurs at the higher deposition temperature of 500...700 °C when the coating consists of not less than 5 elements. It was established that based on the conditions for an electron-beam process of materials formation, the results obtained can be divided into two types: those determined by the condition of evaporation of the target and those determined by the conditions of coating deposition. The density of flows of elements, evaporated from the target, is determined by their specific heat of evaporation. However, the ratio of atoms in the flow, derived in this way, may not be retained in the formed coating due to the secondary evaporation of elements from the growth surface. The obtained results allow us to substantiate principles for the selection of components for achieving the optimal element and phase compositions of high-entropy alloys.На основі аналізу елементного складу матеріалів мішені з високоентропійного шестиелементного сплаву AlTiVCrNbMo і електронно-променевих покриттів на його основі встановлено критичний параметр (питома теплота випаровування елемента), який визначає селективну зміну елементного складу. Показано, що формування однофазного покриття високоентропійного сплаву відбувається, коли до складу покриття входить не менше 5 елементів. Отримані результати дозволяють обґрунтувати принципи підбору компонент для досягнення оптимальних елементного та фазового складу високоентропійних сплавів

Approximation with Random Bases: Pro et Contra

In this work we discuss the problem of selecting suitable approximators from families of parameterized elementary functions that are known to be dense in a Hilbert space of functions. We consider and analyze published procedures, both randomized and deterministic, for selecting elements from these families that have been shown to ensure the rate of convergence in $L_2$ norm of order $O(1/N)$, where $N$ is the number of elements. We show that both randomized and deterministic procedures are successful if additional information about the families of functions to be approximated is provided. In the absence of such additional information one may observe exponential growth of the number of terms needed to approximate the function and/or extreme sensitivity of the outcome of the approximation to parameters. Implications of our analysis for applications of neural networks in modeling and control are illustrated with examples.Comment: arXiv admin note: text overlap with arXiv:0905.067