Radial Basis Function Neural Networks : A Review

Radial Basis Function neural networks (RBFNNs) represent an attractive alternative to other neural network models. One reason is that they form a unifying link between function approximation, regularization, noisy interpolation, classification and density estimation. It is also the case that training RBF neural networks is faster than training multi-layer perceptron networks. RBFNN learning is usually split into an unsupervised part, where center and widths of the Gaussian basis functions are set, and a linear supervised part for weight computation. This paper reviews various learning methods for determining centers, widths, and synaptic weights of RBFNN. In addition, we will point to some applications of RBFNN in various fields. In the end, we name software that can be used for implementing RBFNNs

Separable Gaussian Neural Networks: Structure, Analysis, and Function Approximations

The Gaussian-radial-basis function neural network (GRBFNN) has been a popular choice for interpolation and classification. However, it is computationally intensive when the dimension of the input vector is high. To address this issue, we propose a new feedforward network - Separable Gaussian Neural Network (SGNN) by taking advantage of the separable property of Gaussian functions, which splits input data into multiple columns and sequentially feeds them into parallel layers formed by uni-variate Gaussian functions. This structure reduces the number of neurons from O(N^d) of GRBFNN to O(dN), which exponentially improves the computational speed of SGNN and makes it scale linearly as the input dimension increases. In addition, SGNN can preserve the dominant subspace of the Hessian matrix of GRBFNN in gradient descent training, leading to a similar level of accuracy to GRBFNN. It is experimentally demonstrated that SGNN can achieve 100 times speedup with a similar level of accuracy over GRBFNN on tri-variate function approximations. The SGNN also has better trainability and is more tuning-friendly than DNNs with RuLU and Sigmoid functions. For approximating functions with complex geometry, SGNN can lead to three orders of magnitude more accurate results than a RuLU-DNN with twice the number of layers and the number of neurons per layer

Ensemble Support Vector Machine Models of Radiation-Induced Lung Injury Risk

Patients undergoing radiation therapy can develop a potentially fatal inflammation of the lungs known as radiation pneumonitis: RP). In practice, modeling RP factors is difficult because existing data are under-sampled and imbalanced. Support vector machines: SVMs), a class of statistical learning methods that implicitly maps data into a higher dimensional space, is one machine learning method that recently has been applied to the RP problem with encouraging results. In this thesis, we present and evaluate an ensemble SVM method of modeling radiation pneumonitis. The method internalizes kernel/model parameter selection into model building and enables feature scaling via Olivier Chapelle\u27s method. We show that the ensemble method provides statistically significant increases to the cross-folded area under the receiver operating characteristic curve while maintaining model parsimony. Finally, we extend our model with John C. Platt\u27s method to support non-binary outcomes in order to augment clinical relevancy

Human Interaction Recognition with Audio and Visual Cues

The automated recognition of human activities from video is a fundamental problem with applications in several areas, ranging from video surveillance, and robotics, to smart healthcare, and multimedia indexing and retrieval, just to mention a few. However, the pervasive diffusion of cameras capable of recording audio also makes available to those applications a complementary modality. Despite the sizable progress made in the area of modeling and recognizing group activities, and actions performed by people in isolation from video, the availability of audio cues has rarely being leveraged. This is even more so in the area of modeling and recognizing binary interactions between humans, where also the use of video has been limited.;This thesis introduces a modeling framework for binary human interactions based on audio and visual cues. The main idea is to describe an interaction with a spatio-temporal trajectory modeling the visual motion cues, and a temporal trajectory modeling the audio cues. This poses the problem of how to fuse temporal trajectories from multiple modalities for the purpose of recognition. We propose a solution whereby trajectories are modeled as the output of kernel state space models. Then, we developed kernel-based methods for the audio-visual fusion that act at the feature level, as well as at the kernel level, by exploiting multiple kernel learning techniques. The approaches have been extensively tested and evaluated with a dataset made of videos obtained from TV shows and Hollywood movies, containing five different interactions. The results show the promise of this approach by producing a significant improvement of the recognition rate when audio cues are exploited, clearly setting the state-of-the-art in this particular application

Investigation of Process-Structure Relationship for Additive Manufacturing with Multiphysics Simulation and Physics-Constrained Machine Learning

Metal additive manufacturing (AM) is a group of processes by which metal parts are built layer by layer from powder or wire feedstock with high-energy laser or electron beams. The most well-known metal AM processes include selective laser melting, electron beam melting, and direct energy deposition. Metal AM can significantly improve the manufacturability of products with complex geometries and heterogeneous materials. It has the potential to be widely applied in various industries including automotive, aerospace, biomedical, energy, and other high-value low-volume manufacturing environments. However, the lack of complete and reliable process-structure-property (P-S-P) relationships for metal AM is still the bottleneck to produce defect-free, structurally sound, and reliable AM parts. There are several technical challenges in establishing the P-S-P relationships for process design and optimization. First, there is a lack of fundamental understanding of the rapid solidification process during which microstructures are formed and the properties of solid parts are determined. Second, the curse of dimensionality in the process and structure design space leads to the lack of data to construct reliable P-S-P relationships. Simulation becomes an important tool to enable us to understand rapid solidification given the limitations of experimental techniques for in-situ measurement. In this research, a mesoscale multiphysics simulation model, called phase-field and thermal lattice Boltzmann method (PF-TLBM), is developed with simultaneous considerations of heterogeneous nucleation, solute transport, heat transfer, and phase transition. The simulation can reveal the complex dynamics of rapid solidification in the melt pool, such as the effects of latent heat and cooling rate on dendritic morphology and solute distribution. The microstructure evolution in the complex heating and cooling environment in the layer-by-layer AM process is simulated with the PF-TLBM model. To meet the lack-of-data challenge in constructing P-S-P relationships, a new scheme of multi-fidelity physics-constrained neural network (MF-PCNN) is developed to improve the efficiency of training in neural networks by reducing the required amount of training data and incorporating physical knowledge as constraints. Neural networks with two levels of fidelities are combined to improve prediction accuracy. Low-fidelity networks predict the general trend, whereas high-fidelity networks model local details and fluctuations. The developed MF-PCNN is applied to predict phase transition and dendritic growth. A new physics-constrained neural network with the minimax architecture (PCNN-MM) is also developed, where the training of PCNN-MM is formulated as a minimax problem. A novel training algorithm called Dual-Dimer method is developed to search high-order saddle points. The developed PCNN-MM is also extended to solve multiphysics problems. A new sequential training scheme is developed for PCNN-MMs to ensure the convergence in solving multiphysics problems. A new Dual-Dimer with compressive sampling (DD-CS) algorithm is also developed to alleviate the curse of dimensionality in searching high-order saddle points during the training. A surrogate model of process-structure relationship for AM is constructed based on the PF-TLBM and PCNN-MM. Based on the surrogate model, multi-objective Bayesian optimization is utilized to search the optimal initial temperature and cooling rate to obtain the desired dendritic area and microsegregation level. The developed PF-TLBM and PCNN-MM provide a systematic and efficient approach to construct P-S-P relationships for AM process design.Ph.D

A Review of Kernel Methods for Feature Extraction in Nonlinear Process Monitoring

Kernel methods are a class of learning machines for the fast recognition of nonlinear patterns in any data set. In this paper, the applications of kernel methods for feature extraction in industrial process monitoring are systematically reviewed. First, we describe the reasons for using kernel methods and contextualize them among other machine learning tools. Second, by reviewing a total of 230 papers, this work has identified 12 major issues surrounding the use of kernel methods for nonlinear feature extraction. Each issue was discussed as to why they are important and how they were addressed through the years by many researchers. We also present a breakdown of the commonly used kernel functions, parameter selection routes, and case studies. Lastly, this review provides an outlook into the future of kernel-based process monitoring, which can hopefully instigate more advanced yet practical solutions in the process industries

Modeling and Recognizing Binary Human Interactions

Recognizing human activities from video is an important step forward towards the long-term goal of performing scene understanding fully automatically. Applications in this domain include, but are not limited to, the automated analysis of video surveillance footage for public and private monitoring, remote patient and elderly home monitoring, video archiving, search and retrieval, human-computer interaction, and robotics. While recent years have seen a concentration of works focusing on modeling and recognizing either group activities, or actions performed by people in isolation, modeling and recognizing binary human-human interactions is a fundamental building block that only recently has started to catalyze the attention of researchers.;This thesis introduces a new modeling framework for binary human-human interactions. The main idea is to describe interactions with spatio-temporal trajectories. Interaction trajectories can then be modeled as the output of dynamical systems, and recognizing interactions entails designing a suitable way for comparing them. This poses several challenges, starting from the type of information that should be captured by the trajectories, which defines the geometry structure of the output space of the systems. In addition, decision functions performing the recognition should account for the fact that the people interacting do not have a predefined ordering. This work addresses those challenges by carefully designing a kernel-based approach that combines non-linear dynamical system modeling with kernel PCA. Experimental results computed on three recently published datasets, clearly show the promise of this approach, where the classification accuracy, and the retrieval precision are comparable or better than the state-of-the-art

Development on advanced technologies – design and development of cloud computing model

Big Data has been created from virtually everything around us at all times. Every digital media interaction generates data, from computer browsing and online retail to iTunes shopping and Facebook likes. This data is captured from multiple sources, with terrifying speed, volume and variety. But in order to extract substantial value from them, one must possess the optimal processing power, the appropriate analysis tools and, of course, the corresponding skills. The range of data collected by businesses today is almost unreal. According to IBM, more than 2.5 times four million data bytes generated per year, while the amount of data generated increases at such an astonishing rate that 90 % of it has been generated in just the last two years. Big Data have recently attracted substantial interest from both academics and practitioners. Big Data Analytics (BDA) is increasingly becoming a trending practice that many organizations are adopting with the purpose of constructing valuable information from BD. The analytics process, including the deployment and use of BDA tools, is seen by organizations as a tool to improve operational efficiency though it has strategic potential, drive new revenue streams and gain competitive advantages over business rivals. However, there are different types of analytic applications to consider. This paper presents a view of the BD challenges and methods to help to understand the significance of using the Big Data Technologies. This article based on a bibliographic review, on texts published in scientific journals, on relevant research dealing with the big data that have exploded in recent years, as they are increasingly linked to technolog