14,432 research outputs found
FReLU: Flexible Rectified Linear Units for Improving Convolutional Neural Networks
Rectified linear unit (ReLU) is a widely used activation function for deep
convolutional neural networks. However, because of the zero-hard rectification,
ReLU networks miss the benefits from negative values. In this paper, we propose
a novel activation function called \emph{flexible rectified linear unit
(FReLU)} to further explore the effects of negative values. By redesigning the
rectified point of ReLU as a learnable parameter, FReLU expands the states of
the activation output. When the network is successfully trained, FReLU tends to
converge to a negative value, which improves the expressiveness and thus the
performance. Furthermore, FReLU is designed to be simple and effective without
exponential functions to maintain low cost computation. For being able to
easily used in various network architectures, FReLU does not rely on strict
assumptions by self-adaption. We evaluate FReLU on three standard image
classification datasets, including CIFAR-10, CIFAR-100, and ImageNet.
Experimental results show that the proposed method achieves fast convergence
and higher performances on both plain and residual networks
CIFAR-10: KNN-based Ensemble of Classifiers
In this paper, we study the performance of different classifiers on the
CIFAR-10 dataset, and build an ensemble of classifiers to reach a better
performance. We show that, on CIFAR-10, K-Nearest Neighbors (KNN) and
Convolutional Neural Network (CNN), on some classes, are mutually exclusive,
thus yield in higher accuracy when combined. We reduce KNN overfitting using
Principal Component Analysis (PCA), and ensemble it with a CNN to increase its
accuracy. Our approach improves our best CNN model from 93.33% to 94.03%
Predicting Young's Modulus of Glasses with Sparse Datasets using Machine Learning
Machine learning (ML) methods are becoming popular tools for the prediction
and design of novel materials. In particular, neural network (NN) is a
promising ML method, which can be used to identify hidden trends in the data.
However, these methods rely on large datasets and often exhibit overfitting
when used with sparse dataset. Further, assessing the uncertainty in
predictions for a new dataset or an extrapolation of the present dataset is
challenging. Herein, using Gaussian process regression (GPR), we predict
Young's modulus for silicate glasses having sparse dataset. We show that GPR
significantly outperforms NN for sparse dataset, while ensuring no overfitting.
Further, thanks to the nonparametric nature, GPR provides quantitative bounds
for the reliability of predictions while extrapolating. Overall, GPR presents
an advanced ML methodology for accelerating the development of novel functional
materials such as glasses.Comment: 17 pages, 5 figure
- …