Optimization Methods for Inverse Problems

Optimization plays an important role in solving many inverse problems. Indeed, the task of inversion often either involves or is fully cast as a solution of an optimization problem. In this light, the mere non-linear, non-convex, and large-scale nature of many of these inversions gives rise to some very challenging optimization problems. The inverse problem community has long been developing various techniques for solving such optimization tasks. However, other, seemingly disjoint communities, such as that of machine learning, have developed, almost in parallel, interesting alternative methods which might have stayed under the radar of the inverse problem community. In this survey, we aim to change that. In doing so, we first discuss current state-of-the-art optimization methods widely used in inverse problems. We then survey recent related advances in addressing similar challenges in problems faced by the machine learning community, and discuss their potential advantages for solving inverse problems. By highlighting the similarities among the optimization challenges faced by the inverse problem and the machine learning communities, we hope that this survey can serve as a bridge in bringing together these two communities and encourage cross fertilization of ideas.Comment: 13 page

Combinatorial optimization and metaheuristics

Today, combinatorial optimization is one of the youngest and most active areas of discrete mathematics. It is a branch of optimization in applied mathematics and computer science, related to operational research, algorithm theory and computational complexity theory. It sits at the intersection of several fields, including artificial intelligence, mathematics and software engineering. Its increasing interest arises for the fact that a large number of scientific and industrial problems can be formulated as abstract combinatorial optimization problems, through graphs and/or (integer) linear programs. Some of these problems have polynomial-time (“efficient”) algorithms, while most of them are NP-hard, i.e. it is not proved that they can be solved in polynomial-time. Mainly, it means that it is not possible to guarantee that an exact solution to the problem can be found and one has to settle for an approximate solution with known performance guarantees. Indeed, the goal of approximate methods is to find “quickly” (reasonable run-times), with “high” probability, provable “good” solutions (low error from the real optimal solution). In the last 20 years, a new kind of algorithm commonly called metaheuristics have emerged in this class, which basically try to combine heuristics in high level frameworks aimed at efficiently and effectively exploring the search space. This report briefly outlines the components, concepts, advantages and disadvantages of different metaheuristic approaches from a conceptual point of view, in order to analyze their similarities and differences. The two very significant forces of intensification and diversification, that mainly determine the behavior of a metaheuristic, will be pointed out. The report concludes by exploring the importance of hybridization and integration methods

A Survey on Metric Learning for Feature Vectors and Structured Data

The need for appropriate ways to measure the distance or similarity between data is ubiquitous in machine learning, pattern recognition and data mining, but handcrafting such good metrics for specific problems is generally difficult. This has led to the emergence of metric learning, which aims at automatically learning a metric from data and has attracted a lot of interest in machine learning and related fields for the past ten years. This survey paper proposes a systematic review of the metric learning literature, highlighting the pros and cons of each approach. We pay particular attention to Mahalanobis distance metric learning, a well-studied and successful framework, but additionally present a wide range of methods that have recently emerged as powerful alternatives, including nonlinear metric learning, similarity learning and local metric learning. Recent trends and extensions, such as semi-supervised metric learning, metric learning for histogram data and the derivation of generalization guarantees, are also covered. Finally, this survey addresses metric learning for structured data, in particular edit distance learning, and attempts to give an overview of the remaining challenges in metric learning for the years to come.Comment: Technical report, 59 pages. Changes in v2: fixed typos and improved presentation. Changes in v3: fixed typos. Changes in v4: fixed typos and new method

On green routing and scheduling problem

The vehicle routing and scheduling problem has been studied with much interest within the last four decades. In this paper, some of the existing literature dealing with routing and scheduling problems with environmental issues is reviewed, and a description is provided of the problems that have been investigated and how they are treated using combinatorial optimization tools

Attributes of Big Data Analytics for Data-Driven Decision Making in Cyber-Physical Power Systems

Big data analytics is a virtually new term in power system terminology. This concept delves into the way a massive volume of data is acquired, processed, analyzed to extract insight from available data. In particular, big data analytics alludes to applications of artificial intelligence, machine learning techniques, data mining techniques, time-series forecasting methods. Decision-makers in power systems have been long plagued by incapability and weakness of classical methods in dealing with large-scale real practical cases due to the existence of thousands or millions of variables, being time-consuming, the requirement of a high computation burden, divergence of results, unjustifiable errors, and poor accuracy of the model. Big data analytics is an ongoing topic, which pinpoints how to extract insights from these large data sets. The extant article has enumerated the applications of big data analytics in future power systems through several layers from grid-scale to local-scale. Big data analytics has many applications in the areas of smart grid implementation, electricity markets, execution of collaborative operation schemes, enhancement of microgrid operation autonomy, management of electric vehicle operations in smart grids, active distribution network control, district hub system management, multi-agent energy systems, electricity theft detection, stability and security assessment by PMUs, and better exploitation of renewable energy sources. The employment of big data analytics entails some prerequisites, such as the proliferation of IoT-enabled devices, easily-accessible cloud space, blockchain, etc. This paper has comprehensively conducted an extensive review of the applications of big data analytics along with the prevailing challenges and solutions

Study supporting the interim evaluation of the innovation principle. Final Report November 2019

The European Commission has recognised the importance of a more innovation- oriented EU acquis, gradually exploring the ways in which EU rules can support innovation. The ‘innovation principle’ was introduced to ensure that whenever policy is developed, the impact on innovation is fully assessed. However, as further discussed in this Study, the exact contours of the innovation principle have been shaped very gradually within the context of the EU better regulation agenda: originally advocated by industry in the context of the precautionary principle, the innovation principle has gradually been given a more articulate and consistent role, which aims at complementing the precautionary principle by increasing the salience of impacts on innovation during all phases of the policy cycle. This Study presents an evaluation of the current implementation of the innovation principle, limited to two of its three components, i.e. the Research and Innovation Tool included in the Better Regulation Toolbox, and the innovation deals. As a preliminary caveat, it is important to recall that the implementation of the innovation principle is still in its infancy, and thus the Study only represents a very early assessment of the extent to which the innovation principle is being correctly implemented, and whether changes would be required to make the principle more effective and useful in the context of the EU better regulation agenda. The main finding is that the innovation principle has the potential to contribute to the quality and future-proof nature of EU policy, but that significant changes and effort will be needed for this potential to fully materialise. The most evident areas for improvement are related to the lack of a clear legal basis, the lack of a widely acknowledged definition, the lack of awareness among EU officials and stakeholders, and the lack of adequate skills among those that are called to implement the innovation principle. As a result of these problems, the impact of the innovation principle on the innovation-friendliness of the EU acquis has been limited so far. The Commission should clarify in official documents that the Innovation principle does not entail a de- regulatory approach, and is not incompatible with the precautionary principle: this would also help to have the principle fully recognised and endorsed by all EU institutions, as well as by civil society, often concerned with the possible anti-regulatory narrative around the innovation principle in stakeholder discussions. Apart from clarifications, and further dissemination and training, major improvements are possible in the near future, especially if the innovation principle is brought fully in line with the evolving data-driven nature of digital innovation and provides more guidance to the Commission on how to design experimental regulation, including inter alia so-called ‘regulatory sandboxes’. Finally, the Commission should ensure that the innovation principle is given prominence with the transition to the Horizon Europe programme, in particular due to the anticipated launch of ‘missions’ in key domains