GLOBAL OPTIMIZATION METHODS

Training a neural network is a difficult optimization problem because of numerous local minimums. Many global search algorithms have been used to train neural networks. However, local search algorithms are more efficient with computational resources, and therefore numerous random restarts with a local algorithm may be more effective than a global algorithm. This study uses Monte-Carlo simulations to determine the relative efficiency of a local search algorithm to 9 stochastic global algorithms. The computational requirements of the global algorithms are several times higher than the local algorithm and there is little gain in using the global algorithms to train neural networks.Research Methods/ Statistical Methods,

Optimization Methods for Inverse Problems

Optimization plays an important role in solving many inverse problems. Indeed, the task of inversion often either involves or is fully cast as a solution of an optimization problem. In this light, the mere non-linear, non-convex, and large-scale nature of many of these inversions gives rise to some very challenging optimization problems. The inverse problem community has long been developing various techniques for solving such optimization tasks. However, other, seemingly disjoint communities, such as that of machine learning, have developed, almost in parallel, interesting alternative methods which might have stayed under the radar of the inverse problem community. In this survey, we aim to change that. In doing so, we first discuss current state-of-the-art optimization methods widely used in inverse problems. We then survey recent related advances in addressing similar challenges in problems faced by the machine learning community, and discuss their potential advantages for solving inverse problems. By highlighting the similarities among the optimization challenges faced by the inverse problem and the machine learning communities, we hope that this survey can serve as a bridge in bringing together these two communities and encourage cross fertilization of ideas.Comment: 13 page

The Development Of Optimization Methods For Knowledge Base Enrichment Processes

The paper presents the concept of approach to the research and evaluation of the processes of intellectual activity associated with the enrichment of the knowledge base. A feature of the research of the process dynamics is the need of simultaneous consideration of such diverse factors as the complexity of information perception, the presence of the deviations of the response from the standard in the process of reproduction and accounting of the test time.A significant influence on the methods of optimization of the knowledge base enrichment process is exerted by a considerable duration of the task learning process. This causes the use of the multifactor experimental design theory to accelerate the progress towards the optimum.The research results can be used in the development of technologies for efficient knowledge assimilation, automation of skills, and also in the development of expert systems for diagnostics of the processes of intellectual activity