Bounding Optimality Gap in Stochastic Optimization via Bagging: Statistical Efficiency and Stability

We study a statistical method to estimate the optimal value, and the optimality gap of a given solution for stochastic optimization as an assessment of the solution quality. Our approach is based on bootstrap aggregating, or bagging, resampled sample average approximation (SAA). We show how this approach leads to valid statistical confidence bounds for non-smooth optimization. We also demonstrate its statistical efficiency and stability that are especially desirable in limited-data situations, and compare these properties with some existing methods. We present our theory that views SAA as a kernel in an infinite-order symmetric statistic, which can be approximated via bagging. We substantiate our theoretical findings with numerical results

Paints supply chain optimisation

Imperial Users onl

Deep Neural Networks for ECG-Based Pulse Detection during Out-of-Hospital Cardiac Arrest

The automatic detection of pulse during out-of-hospital cardiac arrest (OHCA) is necessary for the early recognition of the arrest and the detection of return of spontaneous circulation (end of the arrest). The only signal available in every single defibrillator and valid for the detection of pulse is the electrocardiogram (ECG). In this study we propose two deep neural network (DNN) architectures to detect pulse using short ECG segments (5 s), i.e., to classify the rhythm into pulseless electrical activity (PEA) or pulse-generating rhythm (PR). A total of 3914 5-s ECG segments, 2372 PR and 1542 PEA, were extracted from 279 OHCA episodes. Data were partitioned patient-wise into training (80%) and test (20%) sets. The first DNN architecture was a fully convolutional neural network, and the second architecture added a recurrent layer to learn temporal dependencies. Both DNN architectures were tuned using Bayesian optimization, and the results for the test set were compared to state-of-the art PR/PEA discrimination algorithms based on machine learning and hand crafted features. The PR/PEA classifiers were evaluated in terms of sensitivity (Se) for PR, specificity (Sp) for PEA, and the balanced accuracy (BAC), the average of Se and Sp. The Se/Sp/BAC of the DNN architectures were 94.1%/92.9%/93.5% for the first one, and 95.5%/91.6%/93.5% for the second one. Both architectures improved the performance of state of the art methods by more than 1.5 points in BAC.This work was supported by: The Spanish Ministerio de Economía y Competitividad, TEC2015-64678-R, jointly with the Fondo Europeo de Desarrollo Regional (FEDER), UPV/EHU via GIU17/031 and the Basque Government through the grant PRE_2018_2_0260

Serial-batch scheduling – the special case of laser-cutting machines

The dissertation deals with a problem in the field of short-term production planning, namely the scheduling of laser-cutting machines. The object of decision is the grouping of production orders (batching) and the sequencing of these order groups on one or more machines (scheduling). This problem is also known in the literature as "batch scheduling problem" and belongs to the class of combinatorial optimization problems due to the interdependencies between the batching and the scheduling decisions. The concepts and methods used are mainly from production planning, operations research and machine learning

Efficient Variational Inference for Hierarchical Models of Images, Text, and Networks

Variational inference provides a general optimization framework to approximate the posterior distributions of latent variables in probabilistic models. Although effective in simple scenarios, variational inference may be inaccurate or infeasible when the data is high-dimensional, the model structure is complicated, or variable relationships are non-conjugate. We propose solutions to these problems through the smart design and leverage of model structures, the rigorous derivation of variational bounds, and the creation of flexible algorithms for various models with rich, non-conjugate dependencies.Concretely, we first design an interpretable generative model for natural images, in which the hundreds of thousands of pixels per image are split into small patches represented by Gaussian mixture models. Through structured variational inference, the evidence lower bound of this model automatically recovers the popular expected patch log-likelihood method for image processing. A nonparametric extension using hierarchical Dirichlet processes further enables self-similarities to be captured and image-specific clusters created during inference, boosting image denoising and inpainting accuracy.Then we move on to text data, and design hierarchical topic graphs that generalize the bipartite noisy-OR models previously used for medical diagnosis. We derive auxiliary bounds to overcome the non-conjugacy of noisy-OR conditionals, and use stochastic variational inference to efficiently train on datasets with hundreds of thousands of documents. We dramatically increase the algorithm speed through a constrained family of variational bounds, so that only the ancestors of the sparse observed tokens of each document need to be considered.Finally, we propose a general-purpose Monte Carlo variational inference strategy that is directly applicable to any model with discrete variables. Compared to REINFORCE-style stochastic gradient updates, our coordinate-ascent updates have lower variance and converge much faster. Compared to auxiliary-variable bounds crafted for each individual model, our algorithm is simpler to derive and may be easily integrated into probabilistic programming languages for broader use. By avoiding auxiliary variables, we also tighten likelihood bounds and increase robustness to local optima. Extensive experiments on real-world models of images, text, and networks illustrate these appealing advantages

Proactive management of uncertainty to improve scheduling robustness in proces industries

Dinamisme, capacitat de resposta i flexibilitat són característiques essencials en el desenvolupament de la societat actual. Les noves tendències de globalització i els avenços en tecnologies de la informació i comunicació fan que s'evolucioni en un entorn altament dinàmic i incert. La incertesa present en tot procés esdevé un factor crític a l'hora de prendre decisions, així com un repte altament reconegut en l'àrea d'Enginyeria de Sistemes de Procés (PSE). En el context de programació de les operacions, els models de suport a la decisió proposats fins ara, així com també software comercial de planificació i programació d'operacions avançada, es basen generalment en dades estimades, assumint implícitament que el programa d'operacions s'executarà sense desviacions. La reacció davant els efectes de la incertesa en temps d'execució és una pràctica habitual, però no sempre resulta efectiva o factible. L'alternativa és considerar la incertesa de forma proactiva, és a dir, en el moment de prendre decisions, explotant el coneixement disponible en el propi sistema de modelització.Davant aquesta situació es plantegen les següents preguntes: què s'entén per incertesa? Com es pot considerar la incertesa en el problema de programació d'operacions? Què s'entén per robustesa i flexibilitat d'un programa d'operacions? Com es pot millorar aquesta robustesa? Quins beneficis comporta? Aquesta tesi respon a aquestes preguntes en el marc d'anàlisis operacionals en l'àrea de PSE. La incertesa es considera no de la forma reactiva tradicional, sinó amb el desenvolupament de sistemes proactius de suport a la decisió amb l'objectiu d'identificar programes d'operació robustos que serveixin com a referència pel nivell inferior de control de planta, així com també per altres centres en un entorn de cadenes de subministrament. Aquest treball de recerca estableix les bases per formalitzar el concepte de robustesa d'un programa d'operacions de forma sistemàtica. Segons aquest formalisme, els temps d'operació i les ruptures d'equip són considerats inicialment com a principals fonts d'incertesa presents a nivell de programació de la producció. El problema es modelitza mitjançant programació estocàstica, desenvolupant-se finalment un entorn d'optimització basat en simulació que captura les múltiples fonts d'incertesa, així com també estratègies de programació d'operacions reactiva, de forma proactiva. La metodologia desenvolupada en el context de programació de la producció s'estén posteriorment per incloure les operacions de transport en sistemes de múltiples entitats i incertesa en els temps de distribució. Amb aquesta perspectiva més àmplia del nivell d'operació s'estudia la coordinació de les activitats de producció i transport, fins ara centrada en nivells estratègic o tàctic. L'estudi final considera l'efecte de la incertesa en la demanda en les decisions de programació de la producció a curt termini. El problema s'analitza des del punt de vista de gestió del risc, i s'avaluen diferents mesures per controlar l'eficiència del sistema en un entorn incert.En general, la tesi posa de manifest els avantatges en reconèixer i modelitzar la incertesa, amb la identificació de programes d'operació robustos capaços d'adaptar-se a un ampli rang de situacions possibles, enlloc de programes d'operació òptims per un escenari hipotètic. La metodologia proposada a nivell d'operació es pot considerar com un pas inicial per estendre's a nivells de decisió estratègics i tàctics. Alhora, la visió proactiva del problema permet reduir el buit existent entre la teoria i la pràctica industrial, i resulta en un major coneixement del procés, visibilitat per planificar activitats futures, així com també millora l'efectivitat de les tècniques reactives i de tot el sistema en general, característiques altament desitjables per mantenir-se actiu davant la globalitat, competitivitat i dinàmica que envolten un procés.Dynamism, responsiveness, and flexibility are essential features in the development of the current society. Globalization trends and fast advances in communication and information technologies make all evolve in a highly dynamic and uncertain environment. The uncertainty involved in a process system becomes a critical problem in decision making, as well as a recognized challenge in the area of Process Systems Engineering (PSE). In the context of scheduling, decision-support models developed up to this point, as well as commercial advanced planning and scheduling systems, rely generally on estimated input information, implicitly assuming that a schedule will be executed without deviations. The reaction to the effects of the uncertainty at execution time becomes a common practice, but it is not always effective or even possible. The alternative is to address the uncertainty proactively, i.e., at the time of reasoning, exploiting the available knowledge in the modeling procedure itself. In view of this situation, the following questions arise: what do we understand for uncertainty? How can uncertainty be considered within scheduling modeling systems? What is understood for schedule robustness and flexibility? How can schedule robustness be improved? What are the benefits? This thesis answers these questions in the context of operational analysis in PSE. Uncertainty is managed not from the traditional reactive viewpoint, but with the development of proactive decision-support systems aimed at identifying robust schedules that serve as a useful guidance for the lower control level, as well as for dependent entities in a supply chain environment. A basis to formalize the concept of schedule robustness is established. Based on this formalism, variable operation times and equipment breakdowns are first considered as the main uncertainties in short-term production scheduling. The problem is initially modeled using stochastic programming, and a simulation-based stochastic optimization framework is finally developed, which captures the multiple sources of uncertainty, as well as rescheduling strategies, proactively. The procedure-oriented system developed in the context of production scheduling is next extended to involve transport scheduling in multi-site systems with uncertain travel times. With this broader operational perspective, the coordination of production and transport activities, considered so far mainly in strategic and tactical analysis, is assessed. The final research point focuses on the effect of demands uncertainty in short-term scheduling decisions. The problem is analyzed from a risk management viewpoint, and alternative measures are assessed and compared to control the performance of the system in the uncertain environment.Overall, this research work reveals the advantages of recognizing and modeling uncertainty, with the identification of more robust schedules able to adapt to a wide range of possible situations, rather than optimal schedules for a hypothetical scenario. The management of uncertainty proposed from an operational perspective can be considered as a first step towards its extension to tactical and strategic levels of decision. The proactive perspective of the problem results in a more realistic view of the process system, and it is a promising way to reduce the gap between theory and industrial practices. Besides, it provides valuable insight on the process, visibility for future activities, as well as it improves the efficiency of reactive techniques and of the overall system, all highly desirable features to remain alive in the global, competitive, and dynamic process environment

Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment

We present a deep neural network-based approach to image quality assessment (IQA). The network is trained end-to-end and comprises ten convolutional layers and five pooling layers for feature extraction, and two fully connected layers for regression, which makes it significantly deeper than related IQA models. Unique features of the proposed architecture are that: 1) with slight adaptations it can be used in a no-reference (NR) as well as in a full-reference (FR) IQA setting and 2) it allows for joint learning of local quality and local weights, i.e., relative importance of local quality to the global quality estimate, in an unified framework. Our approach is purely data-driven and does not rely on hand-crafted features or other types of prior domain knowledge about the human visual system or image statistics. We evaluate the proposed approach on the LIVE, CISQ, and TID2013 databases as well as the LIVE In the wild image quality challenge database and show superior performance to state-of-the-art NR and FR IQA methods. Finally, cross-database evaluation shows a high ability to generalize between different databases, indicating a high robustness of the learned features