Under Physics-Motivated Constraints, Generally-Non-Algorithmic Computational Problems Become Algorithmically Solvable

It is well known that many computational problems are, in general, not algorithmically solvable: e.g., it is not possible to algorithmically decide whether two computable real numbers are equal, and it is not possible to compute the roots of a computable function. We propose to constraint such operations to certain “sets of typical elements” or “sets of random elements”. In our previous papers, we proposed (and analyzed) physics-motivated definitions for these notions. In short, a set T is a set of typical elements if for every definable sequences of sets A nwith A n  ⊇ A n + 1 and ⋂nAn=∅ role= presentation style= box-sizing: border-box; display: inline-table; line-height: normal; letter-spacing: normal; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; padding: 0px; margin: 0px; position: relative; \u3e⋂nAn=∅⋂nAn=∅, there exists an N for which A N  ∩ T = ∅; the definition of a set of random elements with respect to a probability measure P is similar, with the condition ⋂nAn=∅ role= presentation style= box-sizing: border-box; display: inline-table; line-height: normal; letter-spacing: normal; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; padding: 0px; margin: 0px; position: relative; \u3e⋂nAn=∅⋂nAn=∅replaced by a more general condition limnP(An)=0 role= presentation style= box-sizing: border-box; display: inline-table; line-height: normal; letter-spacing: normal; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; padding: 0px; margin: 0px; position: relative; \u3elimnP(An)=0limnP(An)=0. In this paper, we show that if we restrict computations to such typical or random elements, then problems which are non-computable in the general case – like comparing real numbers or finding the roots of a computable function – become computable

Why A Model Produced by Training a Neural Network Is Often More Computationally Efficient than a Nonlinear Regression Model: A Theoretical Explanation

Many real-life dependencies can be reasonably accurately described by linear functions. If we want a more accurate description, we need to take non-linear terms into account. To take nonlinear terms into account, we can either explicitly add quadratic terms to the regression equation, or, alternatively, we can train a neural network with a non-linear activation function. At first glance, regression algorithms lead to simpler expressions, but in practice, often, a trained neural network turns out to be a more computationally efficient way of predicting the corresponding dependence. In this paper, we provide a reasonable explanation for this empirical fact

Constraint Programming and Decision Making

In many application areas, it is necessary to make effective decisions under constraints. Several area-specific techniques are known for such decision problems; however, because these techniques are area-specific, it is not easy to apply each technique to other applications areas. Cross-fertilization between different application areas is one of the main objectives of the annual International Workshops on Constraint Programming and Decision Making. Those workshops, held in the US (El Paso, Texas), in Europe (Lyon, France), and in Asia (Novosibirsk, Russia), from 2008 to 2012, have attracted researchers and practitioners from all over the world. This volume presents extended versions of selected papers from those workshops. These papers deal with all stages of decision making under constraints: (1) formulating the problem of multi-criteria decision making in precise terms, (2) determining when the corresponding decision problem is algorithmically solvable; (3) finding the corresponding algorithms, and making these algorithms as efficient as possible; and (4) taking into account interval, probabilistic, and fuzzy uncertainty inherent in the corresponding decision making problems. The resulting application areas include environmental studies (selecting the best location for a meteorological tower), biology (selecting the most probable evolution history of a species), and engineering (designing the best control for a magnetic levitation train)

Towards Efficient Ways of Estimating Failure Probability of Mechanical Structures under Interval Uncertainty

Whether a structure is stable depends on the values of the parameters θ = (θ1, . . . , θn) which describe the structure and its environment. Usually, we know the limit function g(θ) describing stability: a structure is stable if and only if g(θ) \u3e 0. If we also know the probability distribution on the set of all possible combinations θ, then we can estimate the failure probability P. In practice, we often know that the probability distribution belongs to the known family of distributions (e.g., normal), but we only know the approximate values p˜i of the parameters pi characterizing the actual distribution. Similarly, we know the family of possible limit functions, but we have only approximate estimates of the parameters corresponding to the actual limit function. In many such situations, we know the accuracy of the corresponding approximations; i.e., we know an upper bound Δi for which |p˜i - pi| ≤ Δi. In this case, the only information that we have about the actual (unknown) values of the corresponding parameters pi is that pi is in the interval [p˜i - Δi, p˜i + Δi]. Different values pi from the corresponding intervals lead, in general, to different values of the failure probability P. So, under such interval uncertainty, it is desirable to find the range [P_, P– ]. In this paper, we describe efficient algorithms for computing this range. We also show how to take into account the model inaccuracy, i.e., the fact that the finite-parametric models of the distribution and of the limit function provide only an approximate descriptions of the actual ones

Prediction of Volcanic Eruptions as a Case Study of Predicting Rare Events in Chaotic Systems with Delay

Volcanic eruptions can be disastrous; it is therefore important to be able to predict them as accurately as possible. Theoretically, we can use the general machine learning techniques for such predictions. However, in general, without any prior information, such methods require an unrealistic amount of computation time. It is therefore desirable to look for additional information that would enable us to speed up the corresponding computations. In this paper, we provide an empirical evidence that the volcanic system exhibit chaotic and delayed character. We also show that in general (and in volcanic predictions in particular), we can speed up the corresponding predictions if we take into account chaotic and delayed character of the corresponding system

Use of Machine Learning to Analyze and -- Hopefully -- Predict Volcano Activity

Volcanic eruptions cause significant loss of lives and property around the world each year. Their importance is highlighted by the sheer number of volcanoes for which eruptive activity is probable. These volcanoes are classified as in a state of unrest. The Global Volcano Project maintained by the Smithsonian Institution estimates that approximately 600 volcanoes, many proximal to major urban areas, are currently in this state of unrest. A spectrum of phenomena serve as precursors to eruption, including ground deformation, emission of gases, and seismic activity. The precursors are caused by magma upwelling from the Moho to the shallow (2-5 km) subsurface and magma movement in the volcano conduit immediately preceding eruption. Precursors have in common the fundamental petrologic processes of melt generation in the lithosphere and subsequent magma differentiation. Our ultimate objective is to apply state-of-the-art machine learning techniques to volcano eruption forecasting. In this paper, we applied machine learning techniques to the precursor data, such as the 1999 eruption of Redoubt volcano, Alaska, for which a comprehensive record of precursor activity exists as USGS public domain files and global data bases, such as the Smithsonian Institution Global Volcanology Project and Aerocom (which is part of the HEMCO data base). As a result, we get geophysically meaningful results

Why Rectified Linear Neurons Are Efficient: Symmetry-Based, Complexity-Based, and Fuzzy-Based Explanations

Traditionally, neural networks used a sigmoid activation function. Recently, it turned out that piecewise linear activation functions are much more efficient -- especially in deep learning applications. However, so far, there have been no convincing theoretical explanation for this empirical efficiency. In this paper, we show that, by using different uncertainty techniques, we can come up with several explanations for the efficiency of piecewise linear neural networks. The existence of several different explanations makes us even more confident in our results -- and thus, in the efficiency of piecewise linear activation functions

How to Best Apply Neural Networks in Geosciences: Towards Optimal Averaging in Dropout Training

The main objectives of geosciences is to find the current state of the Earth -- i.e., solve the corresponding inverse problems -- and to use this knowledge for predicting the future events, such as earthquakes and volcanic eruptions. In both inverse and prediction problems, often, machine learning techniques are very efficient, and at present, the most efficient machine learning technique is deep neural training. To speed up this training, the current learning algorithms use dropout techniques: they train several sub-networks on different portions of data, and then average the results. A natural idea is to use arithmetic mean for this averaging , but empirically, geometric mean works much better. In this paper, we provide a theoretical explanation for the empirical efficiency of selecting geometric mean as the averaging in dropout training

A Symmetry-Based Approach to Selecting Membership Functions and Its Relation to Chemical

Abstract—In many practical situations, we encounter physical quantities like time for which there is no fixed starting point for measurements: physical properties do not change if we simply change (shift) the starting point. To describe knowledge about such properties, it is desirable to select membership functions which are similarly shift-invariant. We show that while we cannot require that each membership function is shift-invariant, we can require that the linear space of all linear combinations of given membership functions is shift-invariant. We describe all such shift-invariant families of membership functions, and we show that they are naturally related to the corresponding formulas of chemical kinetics. Index Terms—membership function, symmetry-based approach, chemical kinetic