Convolutional autoencoder-based multimodal one-class classification

One-class classification refers to approaches of learning using data from a single class only. In this paper, we propose a deep learning one-class classification method suitable for multimodal data, which relies on two convolutional autoencoders jointly trained to reconstruct the positive input data while obtaining the data representations in the latent space as compact as possible. During inference, the distance of the latent representation of an input to the origin can be used as an anomaly score. Experimental results using a multimodal macroinvertebrate image classification dataset show that the proposed multimodal method yields better results as compared to the unimodal approach. Furthermore, study the effect of different input image sizes, and we investigate how recently proposed feature diversity regularizers affect the performance of our approach. We show that such regularizers improve performance.Comment: 5 pages, 1 figure, 4 table

Numerical solution and bifurcation analysis of nonlinear partial differential equations with extreme learning machines

We address a new numerical method based on a class of machine learning methods, the so-called Extreme Learning Machines (ELM) with both sigmoidal and radial-basis functions, for the computation of steady-state solutions and the construction of (one-dimensional) bifurcation diagrams of nonlinear partial differential equations (PDEs). For our illustrations, we considered two benchmark problems, namely (a) the one-dimensional viscous Burgers with both homogeneous (Dirichlet) and non-homogeneous boundary conditions, and, (b) the one- and two-dimensional Liouville–Bratu–Gelfand PDEs with homogeneous Dirichlet boundary conditions. For the one-dimensional Burgers and Bratu PDEs, exact analytical solutions are available and used for comparison purposes against the numerical derived solutions. Furthermore, the numerical efficiency (in terms of numerical accuracy, size of the grid and execution times) of the proposed numerical machine-learning method is compared against central finite differences (FD) and Galerkin weighted-residuals finite-element (FEM) methods. We show that the proposed numerical machine learning method outperforms in terms of numerical accuracy both FD and FEM methods for medium to large sized grids, while provides equivalent results with the FEM for low to medium sized grids; both methods (ELM and FEM) outperform the FD scheme. Furthermore, the computational times required with the proposed machine learning scheme were comparable and in particular slightly smaller than the ones required with FE

Hybrid meta-heuristic algorithm based parameter optimization for extreme learning machines classification

Most classification algorithms suffer from manual parameter tuning and it affects the training computational time and accuracy performance. Extreme Learning Machines (ELM) emerged as a fast training machine learning algorithm that eliminates parameter tuning by randomly assigning the input weights and biases, and analytically determining the output weights using Moore Penrose generalized inverse method. However, the randomness assignment, does not guarantee an optimal set of input weights and biases of the hidden neurons. This will lead to ELM instability and local minimum solution. ELM performance also is affected by the network structure especially the number of hidden nodes. Too many hidden neurons will increase the network structure complexity and computational time. While too few hidden neuron numbers will affect the ELM generalization ability and reduce the accuracy. In this study, a heuristic-based ELM (HELM) scheme was designed to secure an optimal ELM structure. The results of HELM were validated with five rule-based hidden neuron selection schemes. Then HELM performance was compared with Support Vector Machine (SVM), k-Nearest Neighbour (KNN), and Classification and Regression Tree (CART) to investigate its relative competitiveness. Secondly, to improve the stability of ELM, the Moth-Flame Optimization algorithm is hybridized with ELM as MFO-ELM. MFO generates moths and optimizes their positions in the search space with a logarithm spiral model to obtain the optimal values of input weights and biases. The optimal weights and biases from the search space were passed into the ELM input space. However, it did not completely solve the problem of been stuck in the local extremum since MFO could not ensure a good balance between the exploration and exploitation of the search space. Thirdly, a co-evolutionary hybrid algorithm of the Cross-Entropy Moth-Flame Optimization Extreme Learning Machines (CEMFO-ELM) scheme was proposed. The hybrid of CE and MFO metaheuristic algorithms ensured a balance of exploration and exploitation in the search space and reduced the possibility of been trapped in the local minima. The performances of these schemes were evaluated on some selected medical datasets from the University of California, Irvine (UCI) machine learning repository, and compared with standard ELM, PSO-ELM, and CSO-ELM. The hybrid MFO-ELM algorithm enhanced the selection of optimal weights and biases for ELM, therefore improved its classification accuracy in a range of 0.4914 - 6.0762%, and up to 8.9390% with the other comparative ELM optimized meta-heuristic algorithms. The convergence curves plot show that the proposed hybrid CEMFO meta-heuristic algorithm ensured a balance between the exploration and exploitation in the search space, thereby improved the stability up to 53.75%. The overall findings showed that the proposed CEMFO-ELM provided better generalization performance on the classification of medical datasets. Thus, CEMFO-ELM is a suitable tool to be used not only in solving medical classification problems but potentially be used in other real-world problems

Non-iterative and Fast Deep Learning: Multilayer Extreme Learning Machines

In the past decade, deep learning techniques have powered many aspects of our daily life, and drawn ever-increasing research interests. However, conventional deep learning approaches, such as deep belief network (DBN), restricted Boltzmann machine (RBM), and convolutional neural network (CNN), suffer from time-consuming training process due to fine-tuning of a large number of parameters and the complicated hierarchical structure. Furthermore, the above complication makes it difficult to theoretically analyze and prove the universal approximation of those conventional deep learning approaches. In order to tackle the issues, multilayer extreme learning machines (ML-ELM) were proposed, which accelerate the development of deep learning. Compared with conventional deep learning, ML-ELMs are non-iterative and fast due to the random feature mapping mechanism. In this paper, we perform a thorough review on the development of ML-ELMs, including stacked ELM autoencoder (ELM-AE), residual ELM, and local receptive field based ELM (ELM-LRF), as well as address their applications. In addition, we also discuss the connection between random neural networks and conventional deep learning