16 research outputs found
Fractional Calculus - Theory and Applications
In recent years, fractional calculus has led to tremendous progress in various areas of science and mathematics. New definitions of fractional derivatives and integrals have been uncovered, extending their classical definitions in various ways. Moreover, rigorous analysis of the functional properties of these new definitions has been an active area of research in mathematical analysis. Systems considering differential equations with fractional-order operators have been investigated thoroughly from analytical and numerical points of view, and potential applications have been proposed for use in sciences and in technology. The purpose of this Special Issue is to serve as a specialized forum for the dissemination of recent progress in the theory of fractional calculus and its potential applications
Integral Transformation, Operational Calculus and Their Applications
The importance and usefulness of subjects and topics involving integral transformations and operational calculus are becoming widely recognized, not only in the mathematical sciences but also in the physical, biological, engineering and statistical sciences. This book contains invited reviews and expository and original research articles dealing with and presenting state-of-the-art accounts of the recent advances in these important and potentially useful subjects
Newton-MR: Inexact Newton Method With Minimum Residual Sub-problem Solver
We consider a variant of inexact Newton Method, called Newton-MR, in which
the least-squares sub-problems are solved approximately using Minimum Residual
method. By construction, Newton-MR can be readily applied for unconstrained
optimization of a class of non-convex problems known as invex, which subsumes
convexity as a sub-class. For invex optimization, instead of the classical
Lipschitz continuity assumptions on gradient and Hessian, Newton-MR's global
convergence can be guaranteed under a weaker notion of joint regularity of
Hessian and gradient. We also obtain Newton-MR's problem-independent local
convergence to the set of minima. We show that fast local/global convergence
can be guaranteed under a novel inexactness condition, which, to our knowledge,
is much weaker than the prior related works. Numerical results demonstrate the
performance of Newton-MR as compared with several other Newton-type
alternatives on a few machine learning problems.Comment: 35 page
Quantum Error Mitigation Relying on Permutation Filtering
Quantum error mitigation (QEM) is a class of promising techniques capable of
reducing the computational error of variational quantum algorithms tailored for
current noisy intermediate-scale quantum computers. The recently proposed
permutation-based methods are practically attractive, since they do not rely on
any a priori information concerning the quantum channels. In this treatise, we
propose a general framework termed as permutation filters, which includes the
existing permutation-based methods as special cases. In particular, we show
that the proposed filter design algorithm always converge to the global
optimum, and that the optimal filters can provide substantial improvements over
the existing permutation-based methods in the presence of narrowband quantum
noise, corresponding to large-depth, high-error-rate quantum circuits
Input Invex Neural Network
In this paper, we present a novel method to constrain invexity on Neural
Networks (NN). Invex functions ensure every stationary point is global minima.
Hence, gradient descent commenced from any point will lead to the global
minima. Another advantage of invexity on NN is to divide data space locally
into two connected sets with a highly non-linear decision boundary by simply
thresholding the output. To this end, we formulate a universal invex function
approximator and employ it to enforce invexity in NN. We call it Input Invex
Neural Networks (II-NN). We first fit data with a known invex function,
followed by modification with a NN, compare the direction of the gradient and
penalize the direction of gradient on NN if it contradicts with the direction
of reference invex function. In order to penalize the direction of the gradient
we perform Gradient Clipped Gradient Penalty (GC-GP). We applied our method to
the existing NNs for both image classification and regression tasks. From the
extensive empirical and qualitative experiments, we observe that our method
gives the performance similar to ordinary NN yet having invexity. Our method
outperforms linear NN and Input Convex Neural Network (ICNN) with a large
margin. We publish our code and implementation details at github.Comment: 20 page
Convergence of Newton-MR under Inexact Hessian Information
Recently, there has been a surge of interest in designing variants of the
classical Newton-CG in which the Hessian of a (strongly) convex function is
replaced by suitable approximations. This is mainly motivated by large-scale
finite-sum minimization problems that arise in many machine learning
applications. Going beyond convexity, inexact Hessian information has also been
recently considered in the context of algorithms such as trust-region or
(adaptive) cubic regularization for general non-convex problems. Here, we do
that for Newton-MR, which extends the application range of the classical
Newton-CG beyond convexity to invex problems. Unlike the convergence analysis
of Newton-CG, which relies on spectrum preserving Hessian approximations in the
sense of L\"{o}wner partial order, our work here draws from matrix perturbation
theory to estimate the distance between the subspaces underlying the exact and
approximate Hessian matrices. Numerical experiments demonstrate a great degree
of resilience to such Hessian approximations, amounting to a highly efficient
algorithm in large-scale problems.Comment: 32 pages, 10 figure
Interpretable Machine Learning for Electro-encephalography
While behavioral, genetic and psychological markers can provide important information about brain health, research in that area over the last decades has much focused on imaging devices such as magnetic resonance tomography (MRI) to provide non-invasive information about cognitive processes. Unfortunately, MRI based approaches, able to capture the slow changes in blood oxygenation levels, cannot capture electrical brain activity which plays out on a time scale up to three orders of magnitude faster. Electroencephalography (EEG), which has been available in clinical settings for over 60 years, is able to measure brain activity based on rapidly changing electrical potentials measured non-invasively on the scalp. Compared to MRI based research into neurodegeneration, EEG based research has, over the last decade, received much less interest from the machine learning community. But generally, EEG in combination with sophisticated machine learning offers great potential such that neglecting this source of information, compared to MRI or genetics, is not warranted. In collaborating with clinical experts, the ability to link any results provided by machine learning to the existing body of research is especially important as it ultimately provides an intuitive or interpretable understanding. Here, interpretable means the possibility for medical experts to translate the insights provided by a statistical model into a working hypothesis relating to brain function. To this end, we propose in our first contribution a method allowing for ultra-sparse regression which is applied on EEG data in order to identify a small subset of important diagnostic markers highlighting the main differences between healthy brains and brains affected by Parkinson's disease. Our second contribution builds on the idea that in Parkinson's disease impaired functioning of the thalamus causes changes in the complexity of the EEG waveforms. The thalamus is a small region in the center of the brain affected early in the course of the disease. Furthermore, it is believed that the thalamus functions as a pacemaker - akin to a conductor of an orchestra - such that changes in complexity are expressed and quantifiable based on EEG. We use these changes in complexity to show their association with future cognitive decline. In our third contribution we propose an extension of archetypal analysis embedded into a deep neural network. This generative version of archetypal analysis allows to learn an appropriate representation where every sample of a data set can be decomposed into a weighted sum of extreme representatives, the so-called archetypes. This opens up an interesting possibility of interpreting a data set relative to its most extreme representatives. In contrast, clustering algorithms describe a data set relative to its most average representatives. For Parkinson's disease, we show based on deep archetypal analysis, that healthy brains produce archetypes which are different from those produced by brains affected by neurodegeneration