Uncertainty in Real-Time Semantic Segmentation on Embedded Systems

Application for semantic segmentation models in areas such as autonomous vehicles and human computer interaction require real-time predictive capabilities. The challenges of addressing real-time application is amplified by the need to operate on resource constrained hardware. Whilst development of real-time methods for these platforms has increased, these models are unable to sufficiently reason about uncertainty present. This paper addresses this by combining deep feature extraction from pre-trained models with Bayesian regression and moment propagation for uncertainty aware predictions. We demonstrate how the proposed method can yield meaningful uncertainty on embedded hardware in real-time whilst maintaining predictive performance.Comment: 6 pages, 3 figure

Piecewise Deterministic Markov Processes for Bayesian Neural Networks

Inference on modern Bayesian Neural Networks (BNNs) often relies on a variational inference treatment, imposing violated assumptions of independence and the form of the posterior. Traditional MCMC approaches avoid these assumptions at the cost of increased computation due to its incompatibility to subsampling of the likelihood. New Piecewise Deterministic Markov Process (PDMP) samplers permit subsampling, though introduce a model specific inhomogenous Poisson Process (IPPs) which is difficult to sample from. This work introduces a new generic and adaptive thinning scheme for sampling from these IPPs, and demonstrates how this approach can accelerate the application of PDMPs for inference in BNNs. Experimentation illustrates how inference with these methods is computationally feasible, can improve predictive accuracy, MCMC mixing performance, and provide informative uncertainty measurements when compared against other approximate inference schemes.Comment: Includes correction to software and corrigendum not

Crowdsourcing digital health measures to predict Parkinson's disease severity: the Parkinson's Disease Digital Biomarker DREAM Challenge

Consumer wearables and sensors are a rich source of data about patients' daily disease and symptom burden, particularly in the case of movement disorders like Parkinson's disease (PD). However, interpreting these complex data into so-called digital biomarkers requires complicated analytical approaches, and validating these biomarkers requires sufficient data and unbiased evaluation methods. Here we describe the use of crowdsourcing to specifically evaluate and benchmark features derived from accelerometer and gyroscope data in two different datasets to predict the presence of PD and severity of three PD symptoms: tremor, dyskinesia, and bradykinesia. Forty teams from around the world submitted features, and achieved drastically improved predictive performance for PD status (best AUROC = 0.87), as well as tremor- (best AUPR = 0.75), dyskinesia- (best AUPR = 0.48) and bradykinesia-severity (best AUPR = 0.95)

Practical uncertainty in neural networks

The adoption of machine learning technologies has grown considerably as the predictive performance of deep learning models continues to improve. Application of these systems for real-world scenarios requires not only raw predictive power, but also informative uncertainty information. Quantifying uncertainty comes at the expense of increased computation and time, and as a result most models do not aim to communicate any such information. This thesis addresses this by proposing practical means to quantify uncertainty in offline scenarios, real-time scenarios, and within existing neural networks not designed within a probabilistic framework

Stochastic Bouncy Particle Sampler for Bayesian Neural Networks

Due to their perceived computational cost, Markov chain Monte Carlo (MCMC) methods have seen little recent application to Bayesian Neural Networks (BNNs). We show here that new continuous time MCMC methods can alleviate this cost, and allow for efficient sampling within BNNs. We propose a simplified version of the Stochastic Bouncy Particle Sampler, making it suitable to perform inference on both dense and convolutional networks. We introduce a new Python package that leverages modern GPU acceleration, allowing for flexible posterior distributions to be found without prohibitive time and compute restrictions

Bayesian neural networks: An introduction and survey

Neural Networks (NNs) have provided state-of-the-art results for many challenging machine learning tasks such as detection, regression and classification across the domains of computer vision, speech recognition and natural language processing. Despite their success, they are often implemented in a frequentist scheme, meaning they are unable to reason about uncertainty in their predictions. This article introduces Bayesian Neural Networks (BNNs) and the seminal research regarding their implementation. Different approximate inference methods are compared, and used to highlight where future research can improve on current methods

Piecewise Deterministic Markov Processes for Bayesian Neural Networks

Inference on modern Bayesian Neural Networks (BNNs) often relies on a variational inference treatment, imposing violated assumptions of independence and the form of the posterior. Traditional MCMC approaches avoid these assumptions at the cost of increased computation due to its incompatibility to subsampling of the likelihood. New Piecewise Deterministic Markov Process (PDMP) samplers permit subsampling, though introduce a model-specific inhomogenous Poisson Process (IPPs) which is difficult to sample from. This work introduces a new generic and adaptive thinning scheme for sampling from these IPPs, and demonstrates how this approach can accelerate the application of PDMPs for inference in BNNs. Experimentation illustrates how inference with these methods is computationally feasible, can improve predictive accuracy, MCMC mixing performance, and provide informative uncertainty measurements when compared against other approximate inference schemes.</p

Fast and robust pushbroom hyperspectral imaging via DMD-based scanning

We describe a new pushbroom hyperspectral imaging device that has no macro moving part. The main components of the proposed hyperspectral imager are a digital micromirror device (DMD), a CMOS image sensor with no filter as the spectral sensor, a CMOS color (RGB) image sensor as the auxiliary image sensor, and a diffraction grating. Using the image sensor pair, the device can simultaneously capture hyperspectral data as well as RGB images of the scene. The RGB images captured by the auxiliary image sensor can facilitate geometric co-registration of the hyperspectral image slices captured by the spectral sensor. In addition, the information discernible from the RGB images can lead to capturing the spectral data of only the regions of interest within the scene. The proposed hyperspectral imaging architecture is costeffective, fast, and robust. It also enables a trade-off between resolution and speed. We have built an initial prototype based on the proposed design. The prototype can capture a hyperspectral image datacube with a spatial resolution of 400×400 pixels and a spectral resolution of 500 bands in less than thirty seconds.</p

An efficient framework for zero-shot sketch-based image retrieval

Zero-shot sketch-based image retrieval (ZS-SBIR) has recently attracted the attention of the computer vision community due to its real-world applications, and the more realistic and challenging setting that it presents over SBIR. ZS-SBIR inherits the main challenges of multiple computer vision problems including content-based Image Retrieval (CBIR), zero-shot learning and domain adaptation. The majority of previous studies using deep neural networks have achieved improved results by either projecting sketch and images into a common low-dimensional space, or transferring knowledge from seen to unseen classes. However, those approaches are trained with complex frameworks composed of multiple deep convolutional neural networks (CNNs) and are dependent on category-level word labels. This increases the requirements for training resources and datasets. In comparison, we propose a simple and efficient framework that does not require high computational training resources, and learns the semantic embedding space from a vision model rather than a language model, as is done by related studies. Furthermore, at training and inference stages our method only uses a single CNN. In this work, a pre-trained ImageNet CNN (i.e., ResNet50) is fine-tuned with three proposed learning objects: domain-balanced quadruplet loss, semantic classification loss, and semantic knowledge preservation loss. The domain-balanced quadruplet and semantic classification losses are introduced to learn discriminative, semantic and domain invariant features by considering ZS-SBIR as an object detection and verification problem. To preserve semantic knowledge learned with ImageNet and exploit it for unseen categories, the semantic knowledge preservation loss is proposed. To reduce computational cost and increase the accuracy of the semantic knowledge distillation process, ground-truth semantic knowledge is prepared in a class-oriented fashion prior to training. Extensive experiments are conducted on three challenging ZS-SBIR datasets: Sketchy Extended, TU-Berlin Extended and QuickDraw Extended. The proposed method achieves state-of-the-art results, and outperforms the majority of related works by a substantial margin

Plant disease detection using hyperspectral imaging

Precision agriculture has enabled significant progress in improving yield outcomes for farmers. Recent progress in sensing and perception promises to further enhance the use of precision agriculture by allowing the detection of plant diseases and pests. When coupled with robotics methods for spatial localisation, early detection of plant diseases will al- low farmers to respond in a timely and localised manner to dis- ease outbreaks and limit crop damage. This paper proposes the use of hyperspectral imaging (VNIR and SWIR) and machine learning techniques for the detection of the Tomato Spotted Wilt Virus (TSWV) in capsicum plants. Discriminatory features are extracted using the full spectrum, a variety of vegetation indices, and probabilistic topic models. These features are used to train classifiers for discriminating between leaves obtained from healthy and inoculated plants. The results show excellent discrimination based on the full spectrum and comparable results based on data-driven probabilistic topic models and the domain vegetation indices. Additionally our results show increasing classification performance as the dimensionality of the features increase.</p