18 research outputs found

    When Deep Learning Meets Polyhedral Theory: A Survey

    Full text link
    In the past decade, deep learning became the prevalent methodology for predictive modeling thanks to the remarkable accuracy of deep neural networks in tasks such as computer vision and natural language processing. Meanwhile, the structure of neural networks converged back to simpler representations based on piecewise constant and piecewise linear functions such as the Rectified Linear Unit (ReLU), which became the most commonly used type of activation function in neural networks. That made certain types of network structure \unicode{x2014}such as the typical fully-connected feedforward neural network\unicode{x2014} amenable to analysis through polyhedral theory and to the application of methodologies such as Linear Programming (LP) and Mixed-Integer Linear Programming (MILP) for a variety of purposes. In this paper, we survey the main topics emerging from this fast-paced area of work, which bring a fresh perspective to understanding neural networks in more detail as well as to applying linear optimization techniques to train, verify, and reduce the size of such networks

    Tree ensemble kernels for Bayesian optimization with known constraints over mixed-feature spaces

    Get PDF
    Tree ensembles can be well-suited for black-box optimization tasks such as algorithm tuning and neural architecture search, as they achieve good predictive performance with little or no manual tuning, naturally handle discrete feature spaces, and are relatively insensitive to outliers in the training data. Two well-known challenges in using tree ensembles for black-box optimization are (i) effectively quantifying model uncertainty for exploration and (ii) optimizing over the piece-wise constant acquisition function. To address both points simultaneously, we propose using the kernel interpretation of tree ensembles as a Gaussian Process prior to obtain model variance estimates, and we develop a compatible optimization formulation for the acquisition function. The latter further allows us to seamlessly integrate known constraints to improve sampling efficiency by considering domain-knowledge in engineering settings and modeling search space symmetries, e.g., hierarchical relationships in neural architecture search. Our framework performs as well as state-of-the-art methods for unconstrained black-box optimization over continuous/discrete features and outperforms competing methods for problems combining mixed-variable feature spaces and known input constraints.Comment: 27 pages, 9 figures, 4 table

    The integration of scheduling and control: Top-down vs. bottom-up

    No full text
    The flexible operation of continuous processes often requires the integration of scheduling and control. This can be achieved by top-down or bottom-up approaches. We compare the two paradigms in-silico using an air separation unit as a benchmark process. To demonstrate the top-down paradigm, we identify data-driven models of the closed-loop process dynamics based on a mechanistic model and use them in scheduling calculations that are performed offline. The resulting target trajectories are passed to a linear model predictive control (LMPC) system and implemented in the process. To demonstrate the bottom-up paradigm, we define an economic nonlinear model predictive control (eNMPC) scheme, which performs dynamic optimization using the full model in closed-loop to directly obtain the control variable profiles to be implemented in the process. We provide implementations of the process model equations as both a gPROMS and a Modelica model to encourage future comparison of approaches for flexible operation, process control, and/or handling disturbances. The performance, advantages, and disadvantages of the two strategies are analyzed using demand-response scenarios with varying levels of fluctuations in electricity prices, as well as considering the cases of known, instantaneous, and completely unknown load changes. The similarities and differences of the two approaches as relevant to flexible operation of continuous processes are discussed. Integrated scheduling and control leverages existing infrastructure and can be immediately applied to real operation tasks. Both operation strategies achieve successful process operation with remarkable economic improvements (up to 8%) compared to constant operation. eNMPC requires more computational resources, and is – at the moment – not implementable in real-time due to maximum optimization times exceeding the controller sampling time. However, eNMPC achieves up to 2.5 times higher operating cost savings compared to the top-down approach, owing in part to the more accurate modeling of key process dynamics

    A Dynamic Optimization Approach to Probabilistic Process Design under Uncertainty

    No full text
    The process industry is moving toward rigorous flowsheet design optimization with modern algorithms. Optimization results are, however, influenced by uncertainty in the parameters of the mathematical models used in the optimization calculation. Parametric uncertainty is typically addressed using scenario-based approaches, whereby the process is optimized for a predetermined, finite set of scenarios representing the statistical properties of the parameters. This paper presents a novel approach for process design under uncertainty. The framework exploits the semi-infinite nature of sequential dynamic optimization, and is based on representing the uncertain parameters as continuous, time-varying disturbance variables acting on a (static) process model over a pseudo-time domain. The parameter uncertainty space is mapped by intersecting, continuous parameter trajectories instead of a limited set of discrete scenarios. We test the proposed strategy on two case studies: a dimethyl ether plant and the Williams-Otto process, demonstrating superior computational performance compared to scenario-based approaches