958 research outputs found
Learning from Structured Data with High Dimensional Structured Input and Output Domain
Structured data is accumulated rapidly in many applications, e.g. Bioinformatics, Cheminformatics, social network analysis, natural language processing and text mining. Designing and analyzing algorithms for handling these large collections of structured data has received significant interests in data mining and machine learning communities, both in the input and output domain. However, it is nontrivial to adopt traditional machine learning algorithms, e.g. SVM, linear regression to structured data. For one thing, the structural information in the input domain and output domain is ignored if applying the normal algorithms to structured data. For another, the major challenge in learning from many high-dimensional structured data is that input/output domain can contain tens of thousands even larger number of features and labels. With the high dimensional structured input space and/or structured output space, learning a low dimensional and consistent structured predictive function is important for both robustness and interpretability of the model. In this dissertation, we will present a few machine learning models that learn from the data with structured input features and structured output tasks. For learning from the data with structured input features, I have developed structured sparse boosting for graph classification, structured joint sparse PCA for anomaly detection and localization. Besides learning from structured input, I also investigated the interplay between structured input and output under the context of multi-task learning. In particular, I designed a multi-task learning algorithms that performs structured feature selection & task relationship Inference. We will demonstrate the applications of these structured models on subgraph based graph classification, networked data stream anomaly detection/localization, multiple cancer type prediction, neuron activity prediction and social behavior prediction. Finally, through my intern work at IBM T.J. Watson Research, I will demonstrate how to leverage structural information from mobile data (e.g. call detail record and GPS data) to derive important places from people's daily life for transit optimization and urban planning
An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos
Videos represent the primary source of information for surveillance
applications and are available in large amounts but in most cases contain
little or no annotation for supervised learning. This article reviews the
state-of-the-art deep learning based methods for video anomaly detection and
categorizes them based on the type of model and criteria of detection. We also
perform simple studies to understand the different approaches and provide the
criteria of evaluation for spatio-temporal anomaly detection.Comment: 15 pages, double colum
Adaptive Uncertainty Estimation via High-Dimensional Testing on Latent Representations
Uncertainty estimation aims to evaluate the confidence of a trained deep
neural network. However, existing uncertainty estimation approaches rely on
low-dimensional distributional assumptions and thus suffer from the high
dimensionality of latent features. Existing approaches tend to focus on
uncertainty on discrete classification probabilities, which leads to poor
generalizability to uncertainty estimation for other tasks. Moreover, most of
the literature requires seeing the out-of-distribution (OOD) data in the
training for better estimation of uncertainty, which limits the uncertainty
estimation performance in practice because the OOD data are typically unseen.
To overcome these limitations, we propose a new framework using data-adaptive
high-dimensional hypothesis testing for uncertainty estimation, which leverages
the statistical properties of the feature representations. Our method directly
operates on latent representations and thus does not require retraining the
feature encoder under a modified objective. The test statistic relaxes the
feature distribution assumptions to high dimensionality, and it is more
discriminative to uncertainties in the latent representations. We demonstrate
that encoding features with Bayesian neural networks can enhance testing
performance and lead to more accurate uncertainty estimation. We further
introduce a family-wise testing procedure to determine the optimal threshold of
OOD detection, which minimizes the false discovery rate (FDR). Extensive
experiments validate the satisfactory performance of our framework on
uncertainty estimation and task-specific prediction over a variety of
competitors. The experiments on the OOD detection task also show satisfactory
performance of our method when the OOD data are unseen in the training. Codes
are available at https://github.com/HKU-MedAI/bnn_uncertainty.Comment: NeurIPS 202
- …