Unsupervised Deep Transfer Feature Learning for Medical Image Classification

The accuracy and robustness of image classification with supervised deep learning are dependent on the availability of large-scale, annotated training data. However, there is a paucity of annotated data available due to the complexity of manual annotation. To overcome this problem, a popular approach is to use transferable knowledge across different domains by: 1) using a generic feature extractor that has been pre-trained on large-scale general images (i.e., transfer-learned) but which not suited to capture characteristics from medical images; or 2) fine-tuning generic knowledge with a relatively smaller number of annotated images. Our aim is to reduce the reliance on annotated training data by using a new hierarchical unsupervised feature extractor with a convolutional auto-encoder placed atop of a pre-trained convolutional neural network. Our approach constrains the rich and generic image features from the pre-trained domain to a sophisticated representation of the local image characteristics from the unannotated medical image domain. Our approach has a higher classification accuracy than transfer-learned approaches and is competitive with state-of-the-art supervised fine-tuned methods.Comment: 4 pages, 1 figure, 3 tables, Accepted (Oral) as IEEE International Symposium on Biomedical Imaging 201

Deep neural network model of haptic saliency

Haptic exploration usually involves stereotypical systematic movements that are adapted to the task. Here we tested whether exploration movements are also driven by physical stimulus features. We designed haptic stimuli, whose surface relief varied locally in spatial frequency, height, orientation, and anisotropy. In Experiment 1, participants subsequently explored two stimuli in order to decide whether they were same or different. We trained a variational autoencoder to predict the spatial distribution of touch duration from the surface relief of the haptic stimuli. The model successfully predicted where participants touched the stimuli. It could also predict participants' touch distribution from the stimulus' surface relief when tested with two new groups of participants, who performed a different task (Exp. 2) or explored different stimuli (Exp. 3). We further generated a large number of virtual surface reliefs (uniformly expressing a certain combination of features) and correlated the model's responses with stimulus properties to understand the model's preferences in order to infer which stimulus features were preferentially touched by participants. Our results indicate that haptic exploratory behavior is to some extent driven by the physical features of the stimuli, with e.g. edge-like structures, vertical and horizontal patterns, and rough regions being explored in more detail

Fault diagnosis of a rotor-bearing system under variable rotating speeds using two-stage parameter transfer and infrared thermal images

Current fault diagnosis methods for rotor-bearing system are mostly based on analyzing the vibration signals collected at steady rotating speeds. In those methods, the data collected under one operating condition cannot be accurately used for diagnosis under a different condition. Moreover, in vibration monitoring, installing the necessary sensors will affect the equipment structure and hence the vibration response itself. The present paper proposes a new method based on two-stage parameter transfer and infrared thermal images for fault diagnosis of rotor-bearing system under variable rotating speeds. The method of parameter transfer enables the use of data (or parameters) acquired under one operating condition (called the source domain) to be extended for use in a different operating condition (called the target domain). First, scaled exponential linear unit (SELU) and modified stochastic gradient descent (MSGD) are used to construct an enhanced convolutional neural network (ECNN). Second, a stacked convolutional auto-encoder (CAE) trained based on unlabeled source-domain thermal images is employed to initialize a source-domain ECNN. Third, model parameters from the pre-trained source-domain ECNN are transferred to the target-domain ECNN to adapt to the characteristics of the target domain. The collected thermal images for a rotor-bearing system under variable speeds are used to test the transfer diagnosis performance of the proposed method. The experimental results demonstrate the performance improvement and the advantages of the proposed method

A multi-granularity locally optimal prototype-based approach for classification

Prototype-based approaches generally provide better explainability and are widely used for classification. However, the majority of them suffer from system obesity and lack transparency on complex problems. In this paper, a novel classification approach with a multi-layered system structure self-organized from data is proposed. This approach is able to identify local peaks of multi-modal density derived from static data and filter out more representative ones at multiple levels of granularity acting as prototypes. These prototypes are then optimized to their locally optimal positions in the data space and arranged in layers with meaningful dense links in-between to form pyramidal hierarchies based on the respective levels of granularity accordingly. After being primed offline, the constructed classification model is capable of self-developing continuously from streaming data to self-expend its knowledge base. The proposed approach offers higher transparency and is convenient for visualization thanks to the hierarchical nested architecture. Its system identification process is objective, data-driven and free from prior assumptions on data generation model with user- and problem- specific parameters. Its decision-making process follows the “nearest prototype” principle, and is highly explainable and traceable. Numerical examples on a wide range of benchmark problems demonstrate its high performance

Deep rule-based classifier with human-level performance and characteristics

In this paper, a new type of multilayer rule-based classifier is proposed and applied to image classification problems. The proposed approach is entirely data-driven and fully automatic. It is generic and can be applied to various classification and prediction problems, but in this paper we focus on image processing, in particular. The core of the classifier is a fully interpretable, understandable, self-organised set of IF…THEN… fuzzy rules based on the prototypes autonomously identified by using a one-pass type training process. The classifier can self-evolve and be updated continuously without a full retraining. Due to the prototype-based nature, it is non-parametric; its training process is non-iterative, highly parallelizable and computationally efficient. At the same time, the proposed approach is able to achieve very high classification accuracy on various benchmark datasets surpassing most of the published methods, be comparable with the human abilities. In addition, it can start classification from the first image of each class in the same way as humans do, which makes the proposed classifier suitable for real-time applications. Numerical examples of benchmark image processing demonstrate the merits of the proposed approach

Deep neural network model of haptic saliency

Haptic exploration usually involves stereotypical systematic movements that are adapted to the task. Here we tested whether exploration movements are also driven by physical stimulus features. We designed haptic stimuli, whose surface relief varied locally in spatial frequency, height, orientation, and anisotropy. In Experiment 1, participants subsequently explored two stimuli in order to decide whether they were same or different. We trained a variational autoencoder to predict the spatial distribution of touch duration from the surface relief of the haptic stimuli. The model successfully predicted where participants touched the stimuli. It could also predict participants’ touch distribution from the stimulus’ surface relief when tested with two new groups of participants, who performed a different task (Exp. 2) or explored different stimuli (Exp. 3). We further generated a large number of virtual surface reliefs (uniformly expressing a certain combination of features) and correlated the model’s responses with stimulus properties to understand the model’s preferences in order to infer which stimulus features were preferentially touched by participants. Our results indicate that haptic exploratory behavior is to some extent driven by the physical features of the stimuli, with e.g. edge-like structures, vertical and horizontal patterns, and rough regions being explored in more detail

Authenticated public key elliptic curve based on deep convolutional neural network for cybersecurity image encryption application

The demand for cybersecurity is growing to safeguard information flow and enhance data privacy. This essay suggests a novel authenticated public key elliptic curve based on a deep convolutional neural network (APK-EC-DCNN) for cybersecurity image encryption application. The public key elliptic curve discrete logarithmic problem (EC-DLP) is used for elliptic curve Diffie–Hellman key exchange (EC-DHKE) in order to generate a shared session key, which is used as the chaotic system’s beginning conditions and control parameters. In addition, the authenticity and confidentiality can be archived based on ECC to share the (Formula presented.) parameters between two parties by using the EC-DHKE algorithm. Moreover, the 3D Quantum Chaotic Logistic Map (3D QCLM) has an extremely chaotic behavior of the bifurcation diagram and high Lyapunov exponent, which can be used in high-level security. In addition, in order to achieve the authentication property, the secure hash function uses the output sequence of the DCNN and the output sequence of the 3D QCLM in the proposed authenticated expansion diffusion matrix (AEDM). Finally, partial frequency domain encryption (PFDE) technique is achieved by using the discrete wavelet transform in order to satisfy the robustness and fast encryption process. Simulation results and security analysis demonstrate that the proposed encryption algorithm achieved the performance of the state-of-the-art techniques in terms of quality, security, and robustness against noise- and signal-processing attacks

Optimal Model-parameter Determination for Feedforward Artificial Neural Networks

Neural Networks are an immensely versatile tool for state-of-the-art prediction problems. However, they require a training process that involves numerous hyper-parameters. This creates a training process that demands expert knowledge to configure and is often described as a trial-and-error process. The result is a training process that needs to be executed multiple times and this is highly time expensive. Currently, one solution to this problem is to perform a Grid-Search algorithm. This is where a set of possible values (essentially guesses) is declared for each hyper-parameter. Then each combination of hyper-parameters is used to configure the training session. Once the training of each model (hyper-parameter combination) is completed, the best performing model is retained, and the rest are discarded. The problem with this is that it can be wasteful as it explores hyper-parameter combinations that predictably produce poor models. It is also very time consuming and scales poorly with the size of the model. A number of methods are proposed in this {thesis} to efficiently derive hyper-parameters and model parameters and the empirical results are presented. These methods are split into two categories, Weight-Direct Determination (WDD), and Simple Effective Evolutionary Method. The former category exhibits success in certain cases whereas the latter exhibits a broad success across Classification and Regression; amongst a large number of samples and features and small number of samples and features. The thesis concludes that the WDD is only effective on small datasets (both in terms of the number of samples and number of input features). This is due to its dependence on Delaunay Triangulation which exhibits a quadratic time complexity with-respect-to the number of input samples. It is deemed that the WDD methods developed in this research are not optimal for achieving general-purpose application of Multi-Layer Perceptrons. However, the Complete Simple Effective Evolutionary Method (CSEEM) from the SEEM Chapter shows great promise as it is able to perform effectively on the `Knowledge Extraction based on Evolutionary Learning' (KEEL) Datasets for both Regression and Classification. This method can achieve this effectiveness whilst only requiring a single hyper-parameter (the number of children in a population) that is fairly invariant across datasets. In this {thesis}, CSEEM is applied to real-world regression and classification problems. It is also compared to RMSProp (gradient-dependent iterative method) to compare its performance with an existing gradient-dependent method. In both categories, CSEEM consistently performs with a lower normalized square loss and higher classification accuracy, respectively, versus the number hidden nodes when compared to RMSProp