Proximal methods for structured group features and correlation matrix nearness

Tesis doctoral inédita leída en la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Ingeniería Informática. Fecha de lectura: junio de 2014Optimization is ubiquitous in real life as many of the strategies followed both by nature and by humans aim to minimize a certain cost, or maximize a certain benefit. More specifically, numerous strategies in engineering are designed according to a minimization problem, although usually the problems tackled are convex with a di erentiable objective function, since these problems have no local minima and they can be solved with gradient-based techniques. Nevertheless, many interesting problems are not di erentiable, such as, for instance, projection problems or problems based on non-smooth norms. An approach to deal with them can be found in the theory of Proximal Methods (PMs), which are based on iterative local minimizations using the Proximity Operator (ProxOp) of the terms that compose the objective function. This thesis begins with a general introduction and a brief motivation of the work done. The state of the art in PMs is thoroughly reviewed, defining the basic concepts from the very beginning and describing the main algorithms, as far as possible, in a simple and self-contained way. After that, the PMs are employed in the field of supervised regression, where regularized models play a prominent role. In particular, some classical linear sparse models are reviewed and unified under the point of view of regularization, namely the Lasso, the Elastic–Network, the Group Lasso and the Group Elastic–Network. All these models are trained by minimizing an error term plus a regularization term, and thus they fit nicely in the domain of PMs, as the structure of the problem can be exploited by minimizing alternatively the di erent expressions that compose the objective function, in particular using the Fast Iterative Shrinkage–Thresholding Algorithm (FISTA). As a real-world application, it is shown how these models can be used to forecast wind energy, where they yield both good predictions in terms of the error and, more importantly, valuable information about the structure and distribution of the relevant features. Following with the regularized learning approach, a new regularizer is proposed, called the Group Total Variation, which is a group extension of the classical Total Variation regularizer and thus it imposes constancy over groups of features. In order to deal with it, an approach to compute its ProxOp is derived. Moreover, it is shown that this regularizer can be used directly to clean noisy multidimensional signals (such as colour images) or to define a new linear model, the Group Fused Lasso (GFL), which can be then trained using FISTA. It is also exemplified how this model, when applied to regression problems, is able to provide solutions that identify the underlying problem structure. As an additional result of this thesis, a public software implementation of the GFL model is provided. The PMs are also applied to the Nearest Correlation Matrix problem under observation uncertainty. The original problem consists in finding the correlation matrix which is nearest to the true empirical one. Some variants introduce weights to adapt the confidence given to each entry of the matrix; with a more general perspective, in this thesis the problem is explored directly considering uncertainty on the observations, which is formalized as a set of intervals where the measured matrices lie. Two di erent variants are defined under this framework: a robust approach called the Robust Nearest Correlation Matrix (which aims to minimize the worst-case scenario) and an exploratory approach, the Exploratory Nearest Correlation Matrix (which focuses on the best-case scenario). It is shown how both optimization problems can be solved using the Douglas–Rachford PM with a suitable splitting of the objective functions. The thesis ends with a brief overall discussion and pointers to further work.La optimización está presente en todas las facetas de la vida, de hecho muchas de las estrategias tanto de la naturaleza como del ser humano pretenden minimizar un cierto coste, o maximizar un cierto beneficio. En concreto, multitud de estrategias en ingeniería se diseñan según problemas de minimización, que habitualmente son problemas convexos con una función objetivo diferenciable, puesto que en ese caso no hay mínimos locales y los problemas pueden resolverse mediante técnicas basadas en gradiente. Sin embargo, hay muchos problemas interesantes que no son diferenciables, como por ejemplo problemas de proyección o basados en normas no suaves. Una aproximación para abordar estos problemas son los Métodos Proximales (PMs), que se basan en minimizaciones locales iterativas utilizando el Operador de Proximidad (ProxOp) de los términos de la función objetivo. La tesis comienza con una introducción general y una breve motivación del trabajo hecho. Se revisa en profundidad el estado del arte en PMs, definiendo los conceptos básicos y describiendo los algoritmos principales, dentro de lo posible, de forma simple y auto-contenida. Tras ello, se emplean los PMs en el campo de la regresión supervisada, donde los modelos regularizados tienen un papel prominente. En particular, se revisan y unifican bajo esta perspectiva de regularización algunos modelos lineales dispersos clásicos, a saber, Lasso, Elastic–Network, Lasso Grupal y Elastic–Network Grupal. Todos estos modelos se entrenan minimizando un término de error y uno de regularización, y por tanto encajan perfectamente en el dominio de los PMs, ya que la estructura del problema puede ser aprovechada minimizando alternativamente las diferentes expresiones que componen la función objetivo, en particular mediante el Algoritmo Fast Iterative Shrinkage–Thresholding (FISTA). Como aplicación al mundo real, se muestra que estos modelos pueden utilizarse para predecir energía eólica, donde proporcionan tanto buenos resultados en términos del error como información valiosa sobre la estructura y distribución de las características relevantes. Siguiendo con esta aproximación, se propone un nuevo regularizador, llamado Variación Total Grupal, que es una extensión grupal del regularizador clásico de Variación Total y que por tanto induce constancia sobre grupos de características. Para aplicarlo, se desarrolla una aproximación para calcular su ProxOp. Además, se muestra que este regularizador puede utilizarse directamente para limpiar señales multidimensionales ruidosas (como imágenes a color) o para definir un nuevo modelo lineal, el Fused Lasso Grupal (GFL), que se entrena con FISTA. Se ilustra cómo este modelo, cuando se aplica a problemas de regresión, es capaz de proporcionar soluciones que identifican la estructura subyacente del problema. Como resultado adicional de esta tesis, se publica una implementación software del modelo GFL. Asimismo, se aplican los PMs al problema de Matriz de Correlación Próxima (NCM) bajo incertidumbre. El problema original consiste en encontrar la matriz de correlación más cercana a la empírica verdadera. Algunas variantes introducen pesos para ajustar la confianza que se da a cada entrada de la matriz; con un carácter más general, en esta tesis se explora el problema considerando incertidumbre en las observaciones, que se formaliza como un conjunto de intervalos en el que se encuentran las matrices medidas. Bajo este marco se definen dos variantes: una aproximación robusta llamada NCM Robusta (que minimiza el caso peor) y una exploratoria, NCM Exploratoria (que se centra en el caso mejor). Ambos problemas de optimización pueden resolverse con el PM de Douglas–Rachford y una partición adecuada de las funciones objetivo. La tesis concluye con una discusión global y referencias a trabajo futur

Model based forecasting for demand response strategies

The incremental deployment of decentralized renewable energy sources in the distribution grid is triggering a paradigm change for the power sector. This shift from a centralized structure with big power plants to a decentralized scenario of distributed energy resources, such as solar and wind, calls for a more active management of the distribution grid. Conventional distribution grids were passive systems, in which the power was flowing unidirectionally from upstream to downstream. Nowadays, and increasingly in the future, the penetration of distributed generation (DG), with its stochastic nature and lack of controllability, represents a major challenge for the stability of the network, especially at the distribution level. In particular, the power flow reversals produced by DG cause voltage excursions, which must be compensated. This poses an obstacle to the energy transition towards a more sustainable energy mix, which can however be mitigated by using a more active approach towards the control of the distribution networks. Demand side management (DSM) offers a possible solution to the problem, allowing to actively control the balance between generation, consumption and storage, close to the point of generation. An active energy management implies not only the capability to react promptly in case of disturbances, but also to ability to anticipate future events and take control actions accordingly. This is usually achieved through model predictive control (MPC), which requires a prediction of the future disturbances acting on the system. This thesis treat challenges of distributed DSM, with a particular focus on the case of a high penetration of PV power plants. The first subject of the thesis is the evaluation of the performance of models for forecasting and control with low computational requirements, of distributed electrical batteries. The proposed methods are compared by means of closed loop deterministic and stochastic MPC performance. The second subject of the thesis is the development of model based forecasting for PV power plants, and methods to estimate these models without the use of dedicated sensors. The third subject of the thesis concerns strategies for increasing forecasting accuracy when dealing with multiple signals linked by hierarchical relations. Hierarchical forecasting methods are introduced and a distributed algorithm for reconciling base forecasters is presented. At the same time, a new methodology for generating aggregate consistent probabilistic forecasts is proposed. This method can be applied to distributed stochastic DSM, in the presence of high penetration of rooftop installed PV systems. In this case, the forecasts' errors become mutually dependent, raising difficulties in the control problem due to the nontrivial summation of dependent random variables. The benefits of considering dependent forecasting errors over considering them as independent and uncorrelated, are investigated. The last part of the thesis concerns models for distributed energy markets, relying on hierarchical aggregators. To be effective, DSM requires a considerable amount of flexible load and storage to be controllable. This generates the need to be able to pool and coordinate several units, in order to reach a critical mass. In a real case scenario, flexible units will have different owners, who will have different and possibly conflicting interests. In order to recruit as much flexibility as possible, it is therefore importan

Acceleration Methods for Classic Convex Optimization Algorithms

Tesis doctoral inédita leída en la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Ingeniería Informática. Fecha de lectura : 12-09-2017Most Machine Learning models are defined in terms of a convex optimization problem. Thus, developing algorithms to quickly solve such problems its of great interest to the field. We focus in this thesis on two of the most widely used models, the Lasso and Support Vector Machines. The former belongs to the family of regularization methods, and it was introduced in 1996 to perform both variable selection and regression at the same time. This is accomplished by adding a `1-regularization term to the least squares model, achieving interpretability and also a good generalization error. Support Vector Machines were originally formulated to solve a classification problem by finding the maximum-margin hyperplane, that is, the hyperplane which separates two sets of points and its at equal distance from both of them. SVMs were later extended to handle non-separable classes and non-linear classification problems, applying the kernel-trick. A first contribution of this work is to carefully analyze all the existing algorithms to solve both problems, describing not only the theory behind them but also pointing out possible advantages and disadvantages of each one. Although the Lasso and SVMs solve very different problems, we show in this thesis that they are both equivalent. Following a recent result by Jaggi, given an instance of one model we can construct an instance of the other having the same solution, and vice versa. This equivalence allows us to translate theoretical and practical results, such as algorithms, from one field to the other, that have been otherwise being developed independently. We will give in this thesis not only the theoretical result but also a practical application, that consists on solving the Lasso problem using the SMO algorithm, the state-of-the-art solver for non-linear SVMs. We also perform experiments comparing SMO to GLMNet, one of the most popular solvers for the Lasso. The results obtained show that SMO is competitive with GLMNet, and sometimes even faster. Furthermore, motivated by a recent trend where classical optimization methods are being re-discovered in improved forms and successfully applied to many problems, we have also analyzed two classical momentum-based methods: the Heavy Ball algorithm, introduced by Polyak in 1963 and Nesterov’s Accelerated Gradient, discovered by Nesterov in 1983. In this thesis we develop practical versions of Conjugate Gradient, which is essentially equivalent to the Heavy Ball method, and Nesterov’s Acceleration for the SMO algorithm. Experiments comparing the convergence of all the methods are also carried out. The results show that the proposed algorithms can achieve a faster convergence both in terms of iterations and execution time.La mayoría de modelos de Aprendizaje Automático se definen en términos de un problema de optimización convexo. Por tanto, desarrollar algoritmos para resolver rápidamente dichos problemas es de gran interés para este campo. En esta tesis nos centramos en dos de los modelos más usados, Lasso y Support Vector Machines. El primero pertenece a la familia de métodos de regularización, y fue introducido en 1996 para realizar selección de características y regresión al mismo tiempo. Esto se consigue añadiendo una penalización `1al modelo de mínimos cuadrados, obteniendo interpretabilidad y un buen error de generalización. Las Máquinas de Vectores de Soporte fueron formuladas originalmente para resolver un problema de clasificación buscando el hiper-plano de máximo margen, es decir, el hiper-plano que separa los dos conjuntos de puntos y está a la misma distancia de ambos. Las SVMs se han extendido posteriormente para manejar clases no separables y problemas de clasificación no lineales, mediante el uso de núcleos. Una primera contribución de este trabajo es analizar cuidadosamente los algoritmos existentes para resolver ambos problemas, describiendo no solo la teoría detrás de los mismos sino también mencionando las posibles ventajas y desventajas de cada uno. A pesar de que el Lasso y las SVMs resuelven problemas muy diferentes, en esta tesis demostramos que ambos son equivalentes. Continuando con un resultado reciente de Jaggi, dada una instancia de uno de los modelos podemos construir una instancia del otro que tiene la misma solución, y viceversa. Esta equivalencia nos permite trasladar resultados teóricos y prácticos, como por ejemplo algoritmos, de un campo al otro, que se han desarrollado de forma independiente. En esta tesis mostraremos no solo la equivalencia teórica sino también una aplicación práctica, que consiste en resolver el problema Lasso usando el algoritmo SMO, que es el estado del arte para la resolución de SVM no lineales. También realizamos experimentos comparando SMO a GLMNet, uno de los algoritmos más populares para resolver el Lasso. Los resultados obtenidos muestran que SMO es competitivo con GLMNet, y en ocasiones incluso más rápido. Además, motivado por una tendencia reciente donde métodos clásicos de optimización se están re- descubriendo y aplicando satisfactoriamente en muchos problemas, también hemos analizado dos métodos clásicos basados en “momento”: el algoritmo Heavy Ball, creado por Polyak en 1963 y el Gradiente Acelerado de Nesterov, descubierto por Nesterov en 1983. En esta tesis desarrollamos versiones prácticas de Gradiente Conjugado, que es equivalente a Heavy Ball, y Aceleración de Nesterov para el algortimo SMO. Además, también se realizan experimentos comparando todos los métodos. Los resultados muestran que los algoritmos propuestos a menudo convergen más rápido, tanto en términos de iteraciones como de tiempo de ejecución

Earth Observation Open Science and Innovation

geospatial analytics; social observatory; big earth data; open data; citizen science; open innovation; earth system science; crowdsourced geospatial data; citizen science; science in society; data scienc

MS FT-2-2 7 Orthogonal polynomials and quadrature: Theory, computation, and applications

Quadrature rules find many applications in science and engineering. Their analysis is a classical area of applied mathematics and continues to attract considerable attention. This seminar brings together speakers with expertise in a large variety of quadrature rules. It is the aim of the seminar to provide an overview of recent developments in the analysis of quadrature rules. The computation of error estimates and novel applications also are described

Generalized averaged Gaussian quadrature and applications

A simple numerical method for constructing the optimal generalized averaged Gaussian quadrature formulas will be presented. These formulas exist in many cases in which real positive GaussKronrod formulas do not exist, and can be used as an adequate alternative in order to estimate the error of a Gaussian rule. We also investigate the conditions under which the optimal averaged Gaussian quadrature formulas and their truncated variants are internal

XVI Agricultural Science Congress 2023: Transformation of Agri-Food Systems for Achieving Sustainable Development Goals

The XVI Agricultural Science Congress being jointly organized by the National Academy of Agricultural Sciences (NAAS) and the Indian Council of Agricultural Research (ICAR) during 10-13 October 2023, at hotel Le Meridien, Kochi, is a mega event echoing the theme “Transformation of Agri-Food Systems for achieving Sustainable Development Goals”. ICAR-Central Marine Fisheries Research Institute takes great pride in hosting the XVI ASC, which will be the perfect point of convergence of academicians, researchers, students, farmers, fishers, traders, entrepreneurs, and other stakeholders involved in agri-production systems that ensure food and nutritional security for a burgeoning population. With impeding challenges like growing urbanization, increasing unemployment, growing population, increasing food demands, degradation of natural resources through human interference, climate change impacts and natural calamities, the challenges ahead for India to achieve the Sustainable Development Goals (SDGs) set out by the United Nations are many. The XVI ASC will provide an interface for dissemination of useful information across all sectors of stakeholders invested in developing India’s agri-food systems, not only to meet the SDGs, but also to ensure a stable structure on par with agri-food systems around the world. It is an honour to present this Book of Abstracts which is a compilation of a total of 668 abstracts that convey the results of R&D programs being done in India. The abstracts have been categorized under 10 major Themes – 1. Ensuring Food & Nutritional Security: Production, Consumption and Value addition; 2. Climate Action for Sustainable Agri-Food Systems; 3. Frontier Science and emerging Genetic Technologies: Genome, Breeding, Gene Editing; 4. Livestock-based Transformation of Food Systems; 5. Horticulture-based Transformation of Food Systems; 6. Aquaculture & Fisheries-based Transformation of Food Systems; 7. Nature-based Solutions for Sustainable AgriFood Systems; 8. Next Generation Technologies: Digital Agriculture, Precision Farming and AI-based Systems; 9. Policies and Institutions for Transforming Agri-Food Systems; 10. International Partnership for Research, Education and Development. This Book of Abstracts sets the stage for the mega event itself, which will see a flow of knowledge emanating from a zeal to transform and push India’s Agri-Food Systems to perform par excellence and achieve not only the SDGs of the UN but also to rise as a world leader in the sector. I thank and congratulate all the participants who have submitted abstracts for this mega event, and I also applaud the team that has strived hard to publish this Book of Abstracts ahead of the event. I wish all the delegates and participants a very vibrant and memorable time at the XVI ASC