Sparse Regression with Multi-type Regularized Feature Modeling

Within the statistical and machine learning literature, regularization techniques are often used to construct sparse (predictive) models. Most regularization strategies only work for data where all predictors are treated identically, such as Lasso regression for (continuous) predictors treated as linear effects. However, many predictive problems involve different types of predictors and require a tailored regularization term. We propose a multi-type Lasso penalty that acts on the objective function as a sum of subpenalties, one for each type of predictor. As such, we allow for predictor selection and level fusion within a predictor in a data-driven way, simultaneous with the parameter estimation process. We develop a new estimation strategy for convex predictive models with this multi-type penalty. Using the theory of proximal operators, our estimation procedure is computationally efficient, partitioning the overall optimization problem into easier to solve subproblems, specific for each predictor type and its associated penalty. Earlier research applies approximations to non-differentiable penalties to solve the optimization problem. The proposed SMuRF algorithm removes the need for approximations and achieves a higher accuracy and computational efficiency. This is demonstrated with an extensive simulation study and the analysis of a case-study on insurance pricing analytics

Stability and Generalization of ℓp\ell_p-Regularized Stochastic Learning for GCN

Graph convolutional networks (GCN) are viewed as one of the most popular representations among the variants of graph neural networks over graph data and have shown powerful performance in empirical experiments. That $\ell_2$-based graph smoothing enforces the global smoothness of GCN, while (soft) $\ell_1$-based sparse graph learning tends to promote signal sparsity to trade for discontinuity. This paper aims to quantify the trade-off of GCN between smoothness and sparsity, with the help of a general $\ell_p$-regularized $(1<p\leq 2)$ stochastic learning proposed within. While stability-based generalization analyses have been given in prior work for a second derivative objectiveness function, our $\ell_p$-regularized learning scheme does not satisfy such a smooth condition. To tackle this issue, we propose a novel SGD proximal algorithm for GCNs with an inexact operator. For a single-layer GCN, we establish an explicit theoretical understanding of GCN with the $\ell_p$-regularized stochastic learning by analyzing the stability of our SGD proximal algorithm. We conduct multiple empirical experiments to validate our theoretical findings.Comment: Accepted to IJCAI 202

Combinatorial optimization and metaheuristics

Today, combinatorial optimization is one of the youngest and most active areas of discrete mathematics. It is a branch of optimization in applied mathematics and computer science, related to operational research, algorithm theory and computational complexity theory. It sits at the intersection of several fields, including artificial intelligence, mathematics and software engineering. Its increasing interest arises for the fact that a large number of scientific and industrial problems can be formulated as abstract combinatorial optimization problems, through graphs and/or (integer) linear programs. Some of these problems have polynomial-time (“efficient”) algorithms, while most of them are NP-hard, i.e. it is not proved that they can be solved in polynomial-time. Mainly, it means that it is not possible to guarantee that an exact solution to the problem can be found and one has to settle for an approximate solution with known performance guarantees. Indeed, the goal of approximate methods is to find “quickly” (reasonable run-times), with “high” probability, provable “good” solutions (low error from the real optimal solution). In the last 20 years, a new kind of algorithm commonly called metaheuristics have emerged in this class, which basically try to combine heuristics in high level frameworks aimed at efficiently and effectively exploring the search space. This report briefly outlines the components, concepts, advantages and disadvantages of different metaheuristic approaches from a conceptual point of view, in order to analyze their similarities and differences. The two very significant forces of intensification and diversification, that mainly determine the behavior of a metaheuristic, will be pointed out. The report concludes by exploring the importance of hybridization and integration methods

Finish Them!: Pricing Algorithms for Human Computation

Given a batch of human computation tasks, a commonly ignored aspect is how the price (i.e., the reward paid to human workers) of these tasks must be set or varied in order to meet latency or cost constraints. Often, the price is set up-front and not modified, leading to either a much higher monetary cost than needed (if the price is set too high), or to a much larger latency than expected (if the price is set too low). Leveraging a pricing model from prior work, we develop algorithms to optimally set and then vary price over time in order to meet a (a) user-specified deadline while minimizing total monetary cost (b) user-specified monetary budget constraint while minimizing total elapsed time. We leverage techniques from decision theory (specifically, Markov Decision Processes) for both these problems, and demonstrate that our techniques lead to upto 30\% reduction in cost over schemes proposed in prior work. Furthermore, we develop techniques to speed-up the computation, enabling users to leverage the price setting algorithms on-the-fly

A Cluster Elastic Net for Multivariate Regression

We propose a method for estimating coefficients in multivariate regression when there is a clustering structure to the response variables. The proposed method includes a fusion penalty, to shrink the difference in fitted values from responses in the same cluster, and an L1 penalty for simultaneous variable selection and estimation. The method can be used when the grouping structure of the response variables is known or unknown. When the clustering structure is unknown the method will simultaneously estimate the clusters of the response and the regression coefficients. Theoretical results are presented for the penalized least squares case, including asymptotic results allowing for p >> n. We extend our method to the setting where the responses are binomial variables. We propose a coordinate descent algorithm for both the normal and binomial likelihood, which can easily be extended to other generalized linear model (GLM) settings. Simulations and data examples from business operations and genomics are presented to show the merits of both the least squares and binomial methods.Comment: 37 Pages, 11 Figure

Study and application of spectral monitoring techniques for optical network optimization

One of the possible ways to address the constantly increasing amount of heterogeneous and variable internet traffic is the evolution of the current optical networks towards a more flexible, open, and disaggregated paradigm. In such scenarios, the role played by Optical Performance Monitoring (OPM) is fundamental. In fact, OPM allows to balance performance and specification mismatches resulting from the disaggregation adoption and provides the control plane with the necessary feedback to grant the optical networks an adequate automation level. Therefore, new flexible and cost-effective OPM solutions are needed, as well as novel techniques to extract the desired information from the monitored data and process and apply them. In this dissertation, we focus on three aspects related to OPM. We first study a monitoring data plane scheme to acquire the high resolution signal optical spectra in a nonintrusive way. In particular, we propose a coherent detection based Optical Spectrum Analyzer (OSA) enhanced with specific Digital Signal Processing (DSP) to detect spectral slices of the considered optical signals. Then, we identify two main placement strategies for such monitoring solutions, enhancing them using two spectral processing techniques to estimate signal- and optical filter-related parameters. Specifically, we propose a way to estimate the Amplified Spontaneous Emission (ASE) noise or its related Optical Signal-to-Noise (OSNR) using optical spectra acquired at the egress ports of the network nodes and the filter central frequency and 3/6 dB bandwidth, using spectra captured at the ingress ports of the network nodes. To do so, we leverage Machine Learning (ML) algorithms and the function fitting principle, according to the considered scenario. We validate both the monitoring strategies and their related processing techniques through simulations and experiments. The obtained results confirm the validity of the two proposed estimation approaches. In particular, we are able to estimate in-band the OSNR/ASE noise within an egress monitor placement scenario, with a Maximum Absolute Error (MAE) lower than 0.4 dB. Moreover, we are able to estimate the filter central frequency and 3/6 dB bandwidth, within an ingress optical monitor placement scenario, with a MAE lower than 0.5 GHz and 0.98 GHz, respectively. Based on such evaluations, we also compare the two placement scenarios and provide guidelines on their implementation. According to the analysis of specific figures of merit, such as the estimation of the Signal-to-Noise Ratio (SNR) penalty introduced by an optical filter, we identify the ingress monitoring strategy as the most promising. In fact, when compared to scenarios where no monitoring strategy is adopted, the ingress one reduced the SNR penalty estimation by 92%. Finally, we identify a potential application for the monitored information. Specifically, we propose a solution for the optimization of the subchannel spectral spacing in a superchannel. Leveraging convex optimization methods, we implement a closed control loop process for the dynamical reconfiguration of the subchannel central frequencies to optimize specific Quality of Transmission (QoT)-related metrics. Such a solution is based on the information monitored at the superchannel receiver side. In particular, to make all the subchannels feasible, we consider the maximization of the total superchannel capacity and the maximization of the minimum superchannel subchannel SNR value. We validate the proposed approach using simulations, assuming scenarios with different subchannel numbers, signal characteristics, and starting frequency values. The obtained results confirm the effectiveness of our solution. Specifically, compared with the equally spaced subchannel scenario, we are able to improve the total and the minimum subchannel SNR values of a four subchannel superchannel, of 1.45 dB and 1.19 dB, respectively.Una de las posibles formas de hacer frente a la creciente cantidad de tráfico heterogéneo y variable de Internet es la evolución de las actuales redes ópticas hacia un paradigma más flexible, abierto y desagregado. En estos escenarios, el papel que desempeña el modulo óptico de monitorización de prestaciones (OPM) es fundamental. De hecho, el OPM permite equilibrar los desajustes de rendimiento y especificación, los cuales surgen con la adopción de la desagregación; del mismo modo el OPM también proporciona al plano de control la realimentación necesaria para otorgar un nivel de automatización adecuado a las redes ópticas. En esta tesis, nos centramos en tres aspectos relacionados con el OPM. En primer lugar, estudiamos un esquema de monitorización para adquirir, de forma no intrusiva, los espectros ópticos de señales de alta resolución. En concreto, proponemos un analizador de espectro óptico (OSA) basado en detección coherente y mejorado con un específico procesado digital de señal (DSP) para detectar cortes espectrales de las señales ópticas consideradas. A continuación, presentamos dos técnicas de colocación para dichas soluciones de monitorización, mejorándolas mediante dos técnicas de procesamiento espectral para estimar los parámetros relacionados con la señal y el filtro óptico. Específicamente, proponemos un método para estimar el ruido de emisión espontánea amplificada (ASE), o la relación de señal-ruido óptica (OSNR), utilizando espectros ópticos adquiridos en los puertos de salida de los nodos de la red. Del mismo modo, estimamos la frecuencia central del filtro y el ancho de banda de 3/6 dB, utilizando espectros capturados en los puertos de entrada de los nodos de la red. Para ello, aprovechamos los algoritmos de Machine Learning (ML) y el principio de function fitting, según el escenario considerado. Validamos tanto las estrategias de monitorización como las técnicas de procesamiento mediante simulaciones y experimentos. Se puede estimar en banda el ruido ASE/OSNR en un escenario de colocación de monitores de salida, con un Maximum Absolute Error (MAE) inferior a 0.4 dB. Además, se puede estimar la frecuencia central del filtro y el ancho de banda de 3/6 dB, dentro de un escenario de colocación de monitores ópticos de entrada, con un MAE inferior a 0.5 GHz y 0.98 GHz, respectivamente. A partir de estas evaluaciones, también comparamos los dos escenarios de colocación y proporcionamos directrices sobre su aplicación. Según el análisis de específicas figuras de mérito, como la estimación de la penalización de la relación señal-ruido (SNR) introducida por un filtro óptico, demostramos que la estrategia de monitorización de entrada es la más prometedora. De hecho, utilizar un sistema de monitorización de entrada redujo la estimación de la penalización del SNR en un 92%. Por último, identificamos una posible aplicación para la información monitorizada. En concreto, proponemos una solución para la optimización del espaciado espectral de los subcanales en un supercanal. Aprovechando los métodos de optimización convexa, implementamos un proceso cíclico de control cerrado para la reconfiguración dinámica de las frecuencias centrales de los subcanales con el fin de optimizar métricas específicas relacionadas con la calidad de la transmisión (QoT). Esta solución se basa en la información monitorizada en el lado del receptor del supercanal. Validamos el enfoque propuesto mediante simulaciones, asumiendo escenarios con un diferente número de subcanales, distintas características de la señal, y diversos valores de la frecuencia inicial. Los resultados obtenidos confirman la eficacia de nuestra solución. Más específicatamente, en comparación con el escenario de subcanales igualmente espaciados, se pueden mejorar los valores totales y minimos de SNR de los subcanales de un supercanal de cuatro subcanales, de 1.45 dB y 1.19 dB, respectivamentePostprint (published version