Perspectives on Incorporating Expert Feedback into Model Updates

Machine learning (ML) practitioners are increasingly tasked with developing models that are aligned with non-technical experts' values and goals. However, there has been insufficient consideration on how practitioners should translate domain expertise into ML updates. In this paper, we consider how to capture interactions between practitioners and experts systematically. We devise a taxonomy to match expert feedback types with practitioner updates. A practitioner may receive feedback from an expert at the observation- or domain-level, and convert this feedback into updates to the dataset, loss function, or parameter space. We review existing work from ML and human-computer interaction to describe this feedback-update taxonomy, and highlight the insufficient consideration given to incorporating feedback from non-technical experts. We end with a set of open questions that naturally arise from our proposed taxonomy and subsequent survey

Recurrent Deep Neural Networks for Real-Time Sleep Stage Classification From Single Channel EEG

Objective: We investigate the design of deep recurrent neural networks for detecting sleep stages from single channel EEG signals recorded at home by non-expert users. We report the effect of data set size, architecture choices, regularization, and personalization on the classification performance.Methods: We evaluated 58 different architectures and training configurations using three-fold cross validation.Results: A network consisting of convolutional (CONV) layers and long short term memory (LSTM) layers can achieve an agreement with a human annotator of Cohen's Kappa of ~0.73 using a training data set of 19 subjects. Regularization and personalization do not lead to a performance gain.Conclusion: The optimal neural network architecture achieves a performance that is very close to the previously reported human inter-expert agreement of Kappa 0.75.Significance: We give the first detailed account of CONV/LSTM network design process for EEG sleep staging in single channel home based setting

Application of Deep Neural Network in Healthcare data

Biomedical data analysis has been playing an important role in healthcare provision services. For decades, medical practitioners and researchers have been extracting and analyse biomedical data to derive different health-related information. Recently, there has been a significant rise in the amount of biomedical data collection. This is due to the availability of biomedical devices for the extraction of biomedical data which are more portable, easy to use and affordable, as an effect technology advancement. As the amount of biomedical data produced every day increases, the risk of human making analytical and diagnostic mistakes also increases. For example, there are approximately 40 million diagnostic errors involving medical imaging annually worldwide, hence rise a need for the development of fast, accurate, reliable and automatic means for analysis of biomedical data. Conventional machine learning has been used to assist in the analysis and interpretation of biomedical data automatically, but always limited with the need for feature extraction process to train the built models. To achieve this, three studies have been conducted. Two studies were conducted by using EEG signals and one study by using microscopic images of cancer cells. In the first study with EEG signals, our method managed to interpret motor imaginary activities from a 64 channels EEG device with 99% classification accuracy when all the 64 channels were used and 91.5% classification when the number of channels was selected to eight (8) channels. In a second study which involved steady-state visual evoked potential form of EEG signals, our method achieved an average of 94% classification accuracy by using two channels, skin like EEG sensor. In the third study for authentication of cancer cell lines by using microscopic images, our method managed to attain an average of 0.91 F1-score in the authentication of eight classes of cancer cell lines. Studies reported in this thesis, significantly shows that CNN can play a major role in the development of a computerised way in the analysis of biomedical data. Towards provision of better healthcare by using CNN in analysis of different formats of biomedical data, this thesis has three major contributions, i) introduction of a new method for EEG channels selection towards development of portable EEG sensors for real-life application, and ii) introduction of a method for cancer cell lines authentication in the laboratory environment towards development of anti-cancer drugs, and iii) Introduction of a method for authentication of isogenic cancer cell lines

Image Data Augmentation from Small Training Datasets Using Generative Adversarial Networks (GANs)

The scarcity of labelled data is a serious problem since deep models generally require a large amount of training data to achieve desired performance. Data augmentation is widely adopted to enhance the diversity of original datasets and further improve the performance of deep learning models. Learning-based methods, compared to traditional techniques, are specialized in feature extraction, which enhances the effectiveness of data augmentation. Generative adversarial networks (GANs), one of the learning-based generative models, have made remarkable advances in data synthesis. However, GANs still face many challenges in generating high-quality augmented images from small datasets because learning-based generative methods are difficult to create reliable outcomes without sufficient training data. This difficulty deteriorates the data augmentation applications using learning-based methods. In this thesis, to tackle the problem of labelled data scarcity and the training difficulty of augmenting image data from small datasets, three novel GAN models suitable for training with a small number of training samples have been proposed based on three different mapping relationships between the input and output images, including one-to-many mapping, one-to-one mapping, and many-to-many mapping. The proposed GANs employ limited training data, such as a small number of images and limited conditional features, and the synthetic images generated by the proposed GANs are expected to generate images of not only high generative quality but also desirable data diversity. To evaluate the effectiveness of the augmented images generated by the proposed models, inception distances and human perception methods are adopted. Additionally, different image classification tasks were carried out and accuracies from using the original datasets and the augmented datasets were compared. Experimental results illustrate the image classification performance based on convolutional neural networks, i.e., AlexNet, GoogLeNet, ResNet and VGGNet, is comprehensively enhanced, and the scale of improvement is significant when a small number of training samples are involved