Data-driven satisficing measure and ranking

We propose an computational framework for real-time risk assessment and prioritizing for random outcomes without prior information on probability distributions. The basic model is built based on satisficing measure (SM) which yields a single index for risk comparison. Since SM is a dual representation for a family of risk measures, we consider problems constrained by general convex risk measures and specifically by Conditional value-at-risk. Starting from offline optimization, we apply sample average approximation technique and argue the convergence rate and validation of optimal solutions. In online stochastic optimization case, we develop primal-dual stochastic approximation algorithms respectively for general risk constrained problems, and derive their regret bounds. For both offline and online cases, we illustrate the relationship between risk ranking accuracy with sample size (or iterations).Comment: 26 Pages, 6 Figure

Master of Science

thesisThe rotational rheometer (cone-and-plate or parallel plates rheometer) is one of the most effective devices for measuring rheological properties of the viscoelastic liquid: the viscosity (?), the first normal stress difference (N?). However, it has been found practically that some errors were potentially associated with this type of rheometer: The "axial compliance error" is due to the use of linear-variable-displacement-transducer (LVDT) for first normal stress (N?) measurement, and it is potentially significant in the time-dependent material response measurement. Secondly, the low natural frequencies of sensitive LVDT springs fail in recording the high frequency response of a material. Lastly, misalignment of the sample holder (cone and plate) will change the geometry of the sample. These errors were quantified by performing rheology studies with the LVDT detached and a novel device fabricated with Micro-Electronic-Machining-System (MEMS) technique. The device is a pressure sensor plate of 25mm in diameter. It contains eight miniature capacitive pressure sensors, allowing measurements of the radical pressure profile, from which both the first normal stress (N?) and the second normal stress (N?) can be calculated. The apparent response time of N? to start-up of NIST-1490 shear flow was measured. The apparent response time was longer being measured with the LVDT than being measured with the pressure sensor plate, indicating that significant axial compliance errors were present during LVDT measurements. The natural frequency of the LVDT was lower than the high frequency behavior of the tested fluid NIST-1490. A slight cone-plate misalignment, smaller than the manufacturer?s suggested limit, developed a sinusoid-shaped radical pressure profile of the Poly(dimethylsiloxane) (PDMS), corresponding to the axial plane of the tilt. However, this misalignment error can be reduced significantly by averaging the pressure profiles over clockwise and counterclockwise rotation manners. With the pressure sensor plate, the normal stress ratio, ¥= - N?, was measured to be 0.189 for PDMS. ? N

Model and Reinforcement Learning for Markov Games with Risk Preferences

We motivate and propose a new model for non-cooperative Markov game which considers the interactions of risk-aware players. This model characterizes the time-consistent dynamic "risk" from both stochastic state transitions (inherent to the game) and randomized mixed strategies (due to all other players). An appropriate risk-aware equilibrium concept is proposed and the existence of such equilibria is demonstrated in stationary strategies by an application of Kakutani's fixed point theorem. We further propose a simulation-based Q-learning type algorithm for risk-aware equilibrium computation. This algorithm works with a special form of minimax risk measures which can naturally be written as saddle-point stochastic optimization problems, and covers many widely investigated risk measures. Finally, the almost sure convergence of this simulation-based algorithm to an equilibrium is demonstrated under some mild conditions. Our numerical experiments on a two player queuing game validate the properties of our model and algorithm, and demonstrate their worth and applicability in real life competitive decision-making.Comment: 38 pages, 6 tables, 5 figure

Distributional Reinforcement Learning with Online Risk-awareness Adaption

The use of reinforcement learning (RL) in practical applications requires considering sub-optimal outcomes, which depend on the agent's familiarity with the uncertain environment. Dynamically adjusting the level of epistemic risk over the course of learning can tactically achieve reliable optimal policy in safety-critical environments and tackle the sub-optimality of a static risk level. In this work, we introduce a novel framework, Distributional RL with Online Risk Adaption (DRL-ORA), which can quantify the aleatory and epistemic uncertainties compositely and dynamically select the epistemic risk levels via solving a total variation minimization problem online. The risk level selection can be efficiently achieved through grid search using a Follow-The-Leader type algorithm, and its offline oracle is related to "satisficing measure" (in the decision analysis community) under a special modification of the loss function. We show multiple classes of tasks where DRL-ORA outperforms existing methods that rely on either a fixed risk level or manually predetermined risk level adaption. Given the simplicity of our modifications, we believe the framework can be easily incorporated into most RL algorithm variants

Machine Learning-Based Method for Urban Lifeline System Resilience Assessment in GIS*

System resilience, the capability of a system to sustain and recover from deliberate attacks, accidents, or naturally occurring threats or incidents, is a key property to measure the degree of robustness and coupling effect of complex system. The systems of waste disposal, urban water supply, and electricity transmission are typical systems with complex and high coupling features. In this chapter, a methodology for measuring the system resilience of such systems is proposed. It is a process of integrated decision-making which contains two aspects: (1) a five-dimensional indicator framework of system resilience which includes attributes in infrastructural, economic, and social sectors and (2) a hybrid K-means algorithm, which combines entropy theory, bootstrapping, and analytic network process. Through utilizing real data, the methodology can assist to identify and classify the level of system resilience for different geographical regions which are sustained by lifeline systems. The calculation of algorithm, visualization of processed data, and classification of resilience level can be finally realized in geographic information system. Through utilizing by regional governments and local communities, the final result can serve to provide guideline for resource allocation and the prevention of huge economic loss in disasters

Coresets for Wasserstein Distributionally Robust Optimization Problems

Wasserstein distributionally robust optimization (\textsf{WDRO}) is a popular model to enhance the robustness of machine learning with ambiguous data. However, the complexity of \textsf{WDRO} can be prohibitive in practice since solving its ``minimax'' formulation requires a great amount of computation. Recently, several fast \textsf{WDRO} training algorithms for some specific machine learning tasks (e.g., logistic regression) have been developed. However, the research on designing efficient algorithms for general large-scale \textsf{WDRO}s is still quite limited, to the best of our knowledge. \textit{Coreset} is an important tool for compressing large dataset, and thus it has been widely applied to reduce the computational complexities for many optimization problems. In this paper, we introduce a unified framework to construct the $\epsilon$-coreset for the general \textsf{WDRO} problems. Though it is challenging to obtain a conventional coreset for \textsf{WDRO} due to the uncertainty issue of ambiguous data, we show that we can compute a ``dual coreset'' by using the strong duality property of \textsf{WDRO}. Also, the error introduced by the dual coreset can be theoretically guaranteed for the original \textsf{WDRO} objective. To construct the dual coreset, we propose a novel grid sampling approach that is particularly suitable for the dual formulation of \textsf{WDRO}. Finally, we implement our coreset approach and illustrate its effectiveness for several \textsf{WDRO} problems in the experiments