41,686 research outputs found
On the minimax optimality and superiority of deep neural network learning over sparse parameter spaces
Deep learning has been applied to various tasks in the field of machine
learning and has shown superiority to other common procedures such as kernel
methods. To provide a better theoretical understanding of the reasons for its
success, we discuss the performance of deep learning and other methods on a
nonparametric regression problem with a Gaussian noise. Whereas existing
theoretical studies of deep learning have been based mainly on mathematical
theories of well-known function classes such as H\"{o}lder and Besov classes,
we focus on function classes with discontinuity and sparsity, which are those
naturally assumed in practice. To highlight the effectiveness of deep learning,
we compare deep learning with a class of linear estimators representative of a
class of shallow estimators. It is shown that the minimax risk of a linear
estimator on the convex hull of a target function class does not differ from
that of the original target function class. This results in the suboptimality
of linear methods over a simple but non-convex function class, on which deep
learning can attain nearly the minimax-optimal rate. In addition to this
extreme case, we consider function classes with sparse wavelet coefficients. On
these function classes, deep learning also attains the minimax rate up to log
factors of the sample size, and linear methods are still suboptimal if the
assumed sparsity is strong. We also point out that the parameter sharing of
deep neural networks can remarkably reduce the complexity of the model in our
setting.Comment: 33 page
Machine Learning and Integrative Analysis of Biomedical Big Data.
Recent developments in high-throughput technologies have accelerated the accumulation of massive amounts of omics data from multiple sources: genome, epigenome, transcriptome, proteome, metabolome, etc. Traditionally, data from each source (e.g., genome) is analyzed in isolation using statistical and machine learning (ML) methods. Integrative analysis of multi-omics and clinical data is key to new biomedical discoveries and advancements in precision medicine. However, data integration poses new computational challenges as well as exacerbates the ones associated with single-omics studies. Specialized computational approaches are required to effectively and efficiently perform integrative analysis of biomedical data acquired from diverse modalities. In this review, we discuss state-of-the-art ML-based approaches for tackling five specific computational challenges associated with integrative analysis: curse of dimensionality, data heterogeneity, missing data, class imbalance and scalability issues
RMSE-ELM: Recursive Model based Selective Ensemble of Extreme Learning Machines for Robustness Improvement
Extreme learning machine (ELM) as an emerging branch of shallow networks has
shown its excellent generalization and fast learning speed. However, for
blended data, the robustness of ELM is weak because its weights and biases of
hidden nodes are set randomly. Moreover, the noisy data exert a negative
effect. To solve this problem, a new framework called RMSE-ELM is proposed in
this paper. It is a two-layer recursive model. In the first layer, the
framework trains lots of ELMs in different groups concurrently, then employs
selective ensemble to pick out an optimal set of ELMs in each group, which can
be merged into a large group of ELMs called candidate pool. In the second
layer, selective ensemble is recursively used on candidate pool to acquire the
final ensemble. In the experiments, we apply UCI blended datasets to confirm
the robustness of our new approach in two key aspects (mean square error and
standard deviation). The space complexity of our method is increased to some
degree, but the results have shown that RMSE-ELM significantly improves
robustness with slightly computational time compared with representative
methods (ELM, OP-ELM, GASEN-ELM, GASEN-BP and E-GASEN). It becomes a potential
framework to solve robustness issue of ELM for high-dimensional blended data in
the future.Comment: Accepted for publication in Mathematical Problems in Engineering,
09/22/201
- …