PAC-Bayesian Learning of Aggregated Binary Activated Neural Networks with Probabilities over Representations

Considering a probability distribution over parameters is known as an efficient strategy to learn a neural network with non-differentiable activation functions. We study the expectation of a probabilistic neural network as a predictor by itself, focusing on the aggregation of binary activated neural networks with normal distributions over real-valued weights. Our work leverages a recent analysis derived from the PAC-Bayesian framework that derives tight generalization bounds and learning procedures for the expected output value of such an aggregation, which is given by an analytical expression. While the combinatorial nature of the latter has been circumvented by approximations in previous works, we show that the exact computation remains tractable for deep but narrow neural networks, thanks to a dynamic programming approach. This leads us to a peculiar bound minimization learning algorithm for binary activated neural networks, where the forward pass propagates probabilities over representations instead of activation values. A stochastic counterpart that scales to wide architectures is proposed

Optimization as an analysis tool for human complex decision making

We present a problem class of mixed-integer nonlinear programs (MINLPs) with nonconvex continuous relaxations which stem from economic test scenarios that are used in the analysis of human complex problem solving. In a round-based scenario participants hold an executive function. A posteriori a performance indicator is calculated and correlated to personal measures such as intelligence, working memory, or emotion regulation. Altogether, we investigate 2088 optimization problems that differ in size and initial conditions, based on real-world experimental data from 12 rounds of 174 participants. The goals are twofold. First, from the optimal solutions we gain additional insight into a complex system, which facilitates the analysis of a participant’s performance in the test. Second, we propose a methodology to automatize this process by providing a new criterion based on the solution of a series of optimization problems. By providing a mathematical optimization model and this methodology, we disprove the assumption that the “fruit fly of complex problem solving,” the Tailorshop scenario that has been used for dozens of published studies, is not mathematically accessible—although it turns out to be extremely challenging even for advanced state-of-the-art global optimization algorithms and we were not able to solve all instances to global optimality in reasonable time in this study. The publicly available computational tool Tobago [TOBAGO web site https://sourceforge.net/projects/tobago] can be used to automatically generate problem instances of various complexity, contains interfaces to AMPL and GAMS, and is hence ideally suited as a testbed for different kinds of algorithms and solvers. Computational practice is reported with respect to the influence of integer variables, problem dimension, and local versus global optimization with different optimization codes

A Scenario Approach for Operational Planning with Deep Renewables in Power Systems

This work is both enabled by and motivated by the development of new resources and technologies into the power system market operation practice. On one hand, penetration level of uncertain generation resources is constantly increasing and on the other hand, retirement of some of the conventional energy resources like coal power plants makes market operations an attractive topic for both theoretical and state-of-the-art research. In addition, as generation uncertainty increases, it impacts the true cost of energy and causes it to be volatile and on average higher. This work targets flexibility enhancement to the grid to potentially eliminate the impact of uncertainty. Two different viewpoints in two different markets for electricity is targeted. This dissertation looks at the real-time market generation adequacy from the Independent System Operator’s point of view, and the day-ahead scheduling of energy and reserve procurement from the market participant’s point of view. At the real time scale, the emphasis is on developing fast and reliable optimization techniques in solving look-ahead security constrained economic dispatch. The idea is when forecast accuracy gets sharper closer to the real-time and slower power plants retiring in recent years, market participants will spend more and more attention to the real-time market in comparison to the day ahead operation in terms of the energy market. To address it, a data-driven model with rigorous bounds on the risk is proposed. In particular, we formulate the Look-Ahead Security Constrained Economic Dispatch (LAED) problem using the scenario approach techniques. This approach takes historical sample data as input and guarantees a tunable probability of violating the constraints according to the input data size. Scalability of the approach to real power systems was tested on a 2000 bus synthetic grid. The performance of the solution was compared against state-of-the-art deterministic approach as well as a robust approach. Although the real-time market is primarily for energy trading, the day-ahead market is the market for ancillary service trading. In this dissertation, at the day-ahead scale, the focus is on providing ancillary service to the grid by controlling the consumption of millions of privately owned ii pool pumps in the US, while benefiting from energy arbitrage. A conceptual framework, a capacity assessment method, and an operational planning formulation to aggregate flexible loads such as inground swimming pool pumps for a reliable provision of spinning reserve is introduced. Enabled by the Internet of Things (IoT) technologies, many household loads offer tremendous opportunities for aggregated demand response at wholesale level markets. The spinning reserve market is one that fits well in the context of swimming pool pumps in many regions of the U.S. and around the world (e.g. Texas, California, Florida). This work offers rigorous treatment of the collective reliability of many pool pumps as firm generation capacity. Based on the reliability assessment, optimal scheduling of pool pumps is formulated and solved using the deterministic approach and the scenario approach. The case study is performed using empirical data from Electric Reliability Council of Texas (ERCOT). Cost-benefit analysis based on a city suggests the potential business viability of the proposed framework