Dueling Bandits with Qualitative Feedback

We formulate and study a novel multi-armed bandit problem called the qualitative dueling bandit (QDB) problem, where an agent observes not numeric but qualitative feedback by pulling each arm. We employ the same regret as the dueling bandit (DB) problem where the duel is carried out by comparing the qualitative feedback. Although we can naively use classic DB algorithms for solving the QDB problem, this reduction significantly worsens the performance---actually, in the QDB problem, the probability that one arm wins the duel over another arm can be directly estimated without carrying out actual duels. In this paper, we propose such direct algorithms for the QDB problem. Our theoretical analysis shows that the proposed algorithms significantly outperform DB algorithms by incorporating the qualitative feedback, and experimental results also demonstrate vast improvement over the existing DB algorithms

Kernel Single Proxy Control for Deterministic Confounding

We consider the problem of causal effect estimation with an unobserved confounder, where we observe a proxy variable that is associated with the confounder. Although Proxy causal learning (PCL) uses two proxy variables to recover the true causal effect, we show that a single proxy variable is sufficient for causal estimation if the outcome is generated deterministically, generalizing Control Outcome Calibration Approach (COCA). We propose two kernel-based methods for this setting: the first based on the two-stage regression approach, and the second based on a maximum moment restriction approach. We prove that both approaches can consistently estimate the causal effect, and we empirically demonstrate that we can successfully recover the causal effect on challenging synthetic benchmarks

A Neural Mean Embedding Approach for Back-door and Front-door Adjustment

We consider the estimation of average and counterfactual treatment effects, under two settings: back-door adjustment and front-door adjustment. The goal in both cases is to recover the treatment effect without having an access to a hidden confounder. This objective is attained by first estimating the conditional mean of the desired outcome variable given relevant covariates (the "first stage" regression), and then taking the (conditional) expectation of this function as a "second stage" procedure. We propose to compute these conditional expectations directly using a regression function to the learned input features of the first stage, thus avoiding the need for sampling or density estimation. All functions and features (and in particular, the output features in the second stage) are neural networks learned adaptively from data, with the sole requirement that the final layer of the first stage should be linear. The proposed method is shown to converge to the true causal parameter, and outperforms the recent state-of-the-art methods on challenging causal benchmarks, including settings involving high-dimensional image data

A Neural Mean Embedding Approach for Back-door and Front-door Adjustment

We consider the estimation of average and counterfactual treatment effects, under two settings: back-door adjustment and front-door adjustment. The goal in both cases is to recover the treatment effect without having an access to a hidden confounder. This objective is attained by first estimating the conditional mean of the desired outcome variable given relevant covariates (the “first stage” regression), and then taking the (conditional) expectation of this function as a “second stage” procedure. We propose to compute these conditional expectations directly using a regression function to the learned input features of the first stage, thus avoiding the need for sampling or density estimation. All functions and features (and in particular, the output features in the second stage) are neural networks learned adaptively from data, with the sole requirement that the final layer of the first stage should be linear. The proposed method is shown to converge to the true causal parameter, and outperforms the recent state-of-the-art methods on challenging causal benchmarks, including settings involving high-dimensional image data

Using Smartphone Sensors for Localization in BAN

Nowadays, various sensors are embedded in smartphone, making it a great candidate for localization applications. In this chapter, we explored and listed the localization sensors in smartphone, their characteristics, platforms, coordinate system and how they can be used in BAN. These sensors can be roughly divided into three types: physical IMU sensors (accelerometer, gyroscope and magnetometer), virtual IMU (gravity, step counter and electronic compass) and the environmental sensors (barometer, proximity and other miscellaneous). By applying different mathematical methods, the location of the target or the users can be calculated and used for further use, such as navigation, healthcare or military purpose

Intuitionistic Trapezoidal Fuzzy Multiple Criteria Group Decision Making Method Based on Binary Relation

The aim of this paper is to develop a methodology for intuitionistic trapezoidal fuzzy multiple criteria group decision making problems based on binary relation. Firstly, the similarity measure between two vectors based on binary relation is defined, which can be utilized to aggregate preference information. Some desirable properties of the similarity measure based on fuzzy binary relation are also studied. Then, a methodology for fuzzy multiple criteria group decision making is proposed, in which the criteria values are in the terms of intuitionistic trapezoidal fuzzy numbers (ITFNs). Simple and exact formulas are also proposed to determine the vector of the aggregation and group set. According to the weighted expected values of group set, it is easy to rank the alternatives and select the best one. Finally, we apply the proposed method and the Cosine similarity measure method to a numerical example; the numerical results show that our method is effective and practical

Generalized Kernel Ridge Regression for Nonparametric Structural Functions and Semiparametric Treatment Effects

We propose a family of estimators based on kernel ridge regression for nonparametric structural functions (also called dose response curves) and semiparametric treatment effects. Treatment and covariates may be discrete or continuous, and low, high, or infinite dimensional. We reduce causal estimation and inference to combinations of kernel ridge regressions, which have closed form solutions and are easily computed by matrix operations, unlike other machine learning paradigms. This computational simplicity allows us to extend the framework in two directions: from means to increments and distributions of counterfactual outcomes; and from parameters of the full population to those of subpopulations and alternative populations. For structural functions, we prove uniform consistency with finite sample rates. For treatment effects, we prove $\sqrt{n}$ consistency, Gaussian approximation, and semiparametric efficiency with a new double spectral robustness property. We conduct simulations and estimate average, heterogeneous, and incremental structural functions of the US Jobs Corps training program.Comment: Formerly "Kernel Methods for Policy Evaluation: Treatment Effects, Mediation Analysis, and Off-Policy Planning" (2020

Empower Sequence Labeling with Task-Aware Neural Language Model

Linguistic sequence labeling is a general modeling approach that encompasses a variety of problems, such as part-of-speech tagging and named entity recognition. Recent advances in neural networks (NNs) make it possible to build reliable models without handcrafted features. However, in many cases, it is hard to obtain sufficient annotations to train these models. In this study, we develop a novel neural framework to extract abundant knowledge hidden in raw texts to empower the sequence labeling task. Besides word-level knowledge contained in pre-trained word embeddings, character-aware neural language models are incorporated to extract character-level knowledge. Transfer learning techniques are further adopted to mediate different components and guide the language model towards the key knowledge. Comparing to previous methods, these task-specific knowledge allows us to adopt a more concise model and conduct more efficient training. Different from most transfer learning methods, the proposed framework does not rely on any additional supervision. It extracts knowledge from self-contained order information of training sequences. Extensive experiments on benchmark datasets demonstrate the effectiveness of leveraging character-level knowledge and the efficiency of co-training. For example, on the CoNLL03 NER task, model training completes in about 6 hours on a single GPU, reaching F1 score of 91.71$\pm$0.10 without using any extra annotation.Comment: AAAI 201

Minimum observability of probabilistic Boolean networks

This paper studies the minimum observability of probabilistic Boolean networks (PBNs), the main objective of which is to add the fewest measurements to make an unobservable PBN become observable. First of all, the algebraic form of a PBN is established with the help of semi-tensor product (STP) of matrices. By combining the algebraic forms of two identical PBNs into a parallel system, a method to search the states that need to be H-distinguishable is proposed based on the robust set reachability technique. Secondly, a necessary and sufficient condition is given to find the minimum measurements such that a given set can be H-distinguishable. Moreover, by comparing the numbers of measurements for all the feasible H-distinguishable state sets, the least measurements that make the system observable are gained. Finally, an example is given to verify the validity of the obtained results