Decision Making in the Presence of Subjective Stochastic Constraints

Constrained Ranking and Selection considers optimizing a primary performance measure over a finite set of alternatives subject to constraints on secondary performance measures. When the constraints are stochastic, the corresponding performance measures should be estimated by simulation. When the constraints are subjective, the decision maker is willing to consider multiple constraint threshold values. In this thesis, we consider three problem formulations when subjective stochastic constraints are present. In Chapter 2, we consider the problem of finding a set of feasible or near-feasible systems among a finite number of simulated systems in the presence of subjective stochastic constraints. A decision maker may want to test multiple constraint threshold values for the feasibility check, or she may want to determine how a set of feasible systems changes as constraints become more strict with the objective of pruning systems or finding the system with the best performance. We present indifference-zone procedures that recycle observations for the feasibility check and provide an overall probability of correct decision for all threshold values. Our numerical experiments show that the proposed procedures perform well in reducing the required number of observations relative to four alternative procedures (that either restart feasibility check from scratch with respect to each set of thresholds or with the Bonferroni inequality applied in a conservative way) while providing a statistical guarantee on the probability of correct decision. Chapter 3, considers the problem of finding a system with the best primary performance measure among a finite number of simulated systems in the presence of subjective stochastic constraints on secondary performance measures. When no feasible system exists, the decision maker may be willing to relax some constraint thresholds. We take multiple threshold values for each constraint as a user’s input and propose indifference-zone procedures that perform the phases of feasibility check and selection-of-the-best sequentially or simultaneously. We prove that the proposed procedures yield the best system in the most desirable feasible region possible with at least a pre-specified probability. Our experimental results show that our procedures perform well with respect to the number of observations required to make a decision, as compared with straightforward procedures that repeatedly solve the problem for each set of constraint thresholds. In Chapter 4, we consider the problem of finding a portfolio of systems with the best primary performance measure among finitely many simulated systems as stochastic constraints on secondary performance measures are relaxed. By finding a portfolio of the best systems under a variety of constraint thresholds, the decision maker can identify a robust solution with respect to the constraints or consider the trade-off between the primary performance measure and the level of feasibility of the secondary performance measures. We propose indifference-zone procedures that perform the phases of feasibility check and selection-of-the-best sequentially and simultaneously, and prove that the proposed procedures identify the portolio of the best systems with at least a pre-specified probability. Our proposed procedures show a significant reduction in the required number of observations compared with straightforward procedures that repeatedly identify the best system with respect to each set of constraint thresholds.Ph.D

Cost Aware Untargeted Poisoning Attack against Graph Neural Networks,

Graph Neural Networks (GNNs) have become widely used in the field of graph mining. However, these networks are vulnerable to structural perturbations. While many research efforts have focused on analyzing vulnerability through poisoning attacks, we have identified an inefficiency in current attack losses. These losses steer the attack strategy towards modifying edges targeting misclassified nodes or resilient nodes, resulting in a waste of structural adversarial perturbation. To address this issue, we propose a novel attack loss framework called the Cost Aware Poisoning Attack (CA-attack) to improve the allocation of the attack budget by dynamically considering the classification margins of nodes. Specifically, it prioritizes nodes with smaller positive margins while postponing nodes with negative margins. Our experiments demonstrate that the proposed CA-attack significantly enhances existing attack strategie

Dual Co-Matching Network for Multi-choice Reading Comprehension

Multi-choice reading comprehension is a challenging task that requires complex reasoning procedure. Given passage and question, a correct answer need to be selected from a set of candidate answers. In this paper, we propose \textbf{D}ual \textbf{C}o-\textbf{M}atching \textbf{N}etwork (\textbf{DCMN}) which model the relationship among passage, question and answer bidirectionally. Different from existing approaches which only calculate question-aware or option-aware passage representation, we calculate passage-aware question representation and passage-aware answer representation at the same time. To demonstrate the effectiveness of our model, we evaluate our model on a large-scale multiple choice machine reading comprehension dataset (i.e. RACE). Experimental result show that our proposed model achieves new state-of-the-art results.Comment: arXiv admin note: text overlap with arXiv:1806.04068 by other author

Semantics-aware BERT for Language Understanding

The latest work on language representations carefully integrates contextualized features into language model training, which enables a series of success especially in various machine reading comprehension and natural language inference tasks. However, the existing language representation models including ELMo, GPT and BERT only exploit plain context-sensitive features such as character or word embeddings. They rarely consider incorporating structured semantic information which can provide rich semantics for language representation. To promote natural language understanding, we propose to incorporate explicit contextual semantics from pre-trained semantic role labeling, and introduce an improved language representation model, Semantics-aware BERT (SemBERT), which is capable of explicitly absorbing contextual semantics over a BERT backbone. SemBERT keeps the convenient usability of its BERT precursor in a light fine-tuning way without substantial task-specific modifications. Compared with BERT, semantics-aware BERT is as simple in concept but more powerful. It obtains new state-of-the-art or substantially improves results on ten reading comprehension and language inference tasks.Comment: Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-2020

SG-Net: Syntax-Guided Machine Reading Comprehension

For machine reading comprehension, the capacity of effectively modeling the linguistic knowledge from the detail-riddled and lengthy passages and getting ride of the noises is essential to improve its performance. Traditional attentive models attend to all words without explicit constraint, which results in inaccurate concentration on some dispensable words. In this work, we propose using syntax to guide the text modeling by incorporating explicit syntactic constraints into attention mechanism for better linguistically motivated word representations. In detail, for self-attention network (SAN) sponsored Transformer-based encoder, we introduce syntactic dependency of interest (SDOI) design into the SAN to form an SDOI-SAN with syntax-guided self-attention. Syntax-guided network (SG-Net) is then composed of this extra SDOI-SAN and the SAN from the original Transformer encoder through a dual contextual architecture for better linguistics inspired representation. To verify its effectiveness, the proposed SG-Net is applied to typical pre-trained language model BERT which is right based on a Transformer encoder. Extensive experiments on popular benchmarks including SQuAD 2.0 and RACE show that the proposed SG-Net design helps achieve substantial performance improvement over strong baselines.Comment: Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-2020