Design of generalized fractional order gradient descent method

This paper focuses on the convergence problem of the emerging fractional order gradient descent method, and proposes three solutions to overcome the problem. In fact, the general fractional gradient method cannot converge to the real extreme point of the target function, which critically hampers the application of this method. Because of the long memory characteristics of fractional derivative, fixed memory principle is a prior choice. Apart from the truncation of memory length, two new methods are developed to reach the convergence. The one is the truncation of the infinite series, and the other is the modification of the constant fractional order. Finally, six illustrative examples are performed to illustrate the effectiveness and practicability of proposed methods.Comment: 8 pages, 16 figure

Privacy-Preserving and Outsourced Multi-User k-Means Clustering

Many techniques for privacy-preserving data mining (PPDM) have been investigated over the past decade. Often, the entities involved in the data mining process are end-users or organizations with limited computing and storage resources. As a result, such entities may want to refrain from participating in the PPDM process. To overcome this issue and to take many other benefits of cloud computing, outsourcing PPDM tasks to the cloud environment has recently gained special attention. We consider the scenario where n entities outsource their databases (in encrypted format) to the cloud and ask the cloud to perform the clustering task on their combined data in a privacy-preserving manner. We term such a process as privacy-preserving and outsourced distributed clustering (PPODC). In this paper, we propose a novel and efficient solution to the PPODC problem based on k-means clustering algorithm. The main novelty of our solution lies in avoiding the secure division operations required in computing cluster centers altogether through an efficient transformation technique. Our solution builds the clusters securely in an iterative fashion and returns the final cluster centers to all entities when a pre-determined termination condition holds. The proposed solution protects data confidentiality of all the participating entities under the standard semi-honest model. To the best of our knowledge, ours is the first work to discuss and propose a comprehensive solution to the PPODC problem that incurs negligible cost on the participating entities. We theoretically estimate both the computation and communication costs of the proposed protocol and also demonstrate its practical value through experiments on a real dataset.Comment: 16 pages, 2 figures, 5 table

Iterative learning control of integer and noninteger order: An overview

Ovaj rad daje pregledni prikaz nedavno prezentiranih i objavljenih rezultata autora koji se odnose na primenu iterativnog upravljanja putem učenja (ILC) i to celog reda kao i necelog reda. ILC predstavlja jedno od važnih oblasti u teoriji upravljanja i ono je moćan koncept upravljanja koji na iterativan način poboljšava ponašanje procesa koji su po prirodi ponovljivi. ILC je pogodno za upravljanje šire klase mehatroničkih sistema i posebno su pogodni za upravljanje kretanja robotskih sistema koji imaju važnu ulogu u biomehatroničkim, tehničkim sistemima koji uključuju primenu i vojnoj industriju itd. U prvom delu rada predstavljeni su rezultati koji se odnose na primenu višeg celobrojnog reda PD tipa sa pratećom numeričkom simulacijom. Takođe, još jedna druga ILC šema celobrojnog reda je predložena za dati robotski sistem sa tri stepena slobode u rešavanju zadatka praćenja što je i verifikovano kroz simulacioni primer. U drugom delu, predstavljeni su rezultati koji se odnose na primenu ILC frakcionog reda gde je prvo PDα tip predložen za linearni sistem frakcionog reda. Pokazano je da se pod odredjenim dovoljnim uslovima koji uključuju operatore učenja konvergencija datog sistema može biti garantovana. Takodje, PIβDα tip ILC upravljanja je predložen za linearni sistem frakcionog reda sa kašnjenjem. Konačno, dovoljni uslovi za konvergenciju u vremenskom domenu predloženog ILC upravljanja su dati odgovarajućom teoremom sa pratećim dokazom.This paper provides an overview of the recently presented and published results relating to the use of iterative learning control (ILC) based on and integer and fractional order. ILC is one of the recent topics in control theories and it is a powerful control concept that iteratively improves the behavior of processes that are repetitive in nature. ILC is suitable for controlling a wider class of mechatronic systems - it is especially suitable for motion control of robotic systems that attract and hold an important position in biomechatronical, technical systems involving the application, military industry, etc. The first part of the paper presents the results relating to the application of higher integer order PD type ILC with numerical simulation. Also, another integer order ILC scheme is proposed for a given robotic system with three degrees of freedom for task-space trajectory tracking where the effectiveness of the suggested control is demonstrated through a simulation procedure. In the second part, the results related to the application of the fractional order of ILC are presented where PDα type of ILC is proposed firstly, for a fractional order linear time invariant system. It is shown that under some sufficient conditions which include the learning operators, convergence of the learning system can be guaranteed. Also, PIβDα type of ILC is suggested for a fractional order linear time delay system. Finally, sufficient conditions for the convergence in the time domain of the proposed ILC were given by the corresponding theorem together with its proof

Iterative learning control of integer and noninteger order: An overview

Ovaj rad daje pregledni prikaz nedavno prezentiranih i objavljenih rezultata autora koji se odnose na primenu iterativnog upravljanja putem učenja (ILC) i to celog reda kao i necelog reda. ILC predstavlja jedno od važnih oblasti u teoriji upravljanja i ono je moćan koncept upravljanja koji na iterativan način poboljšava ponašanje procesa koji su po prirodi ponovljivi. ILC je pogodno za upravljanje šire klase mehatroničkih sistema i posebno su pogodni za upravljanje kretanja robotskih sistema koji imaju važnu ulogu u biomehatroničkim, tehničkim sistemima koji uključuju primenu i vojnoj industriju itd. U prvom delu rada predstavljeni su rezultati koji se odnose na primenu višeg celobrojnog reda PD tipa sa pratećom numeričkom simulacijom. Takođe, još jedna druga ILC šema celobrojnog reda je predložena za dati robotski sistem sa tri stepena slobode u rešavanju zadatka praćenja što je i verifikovano kroz simulacioni primer. U drugom delu, predstavljeni su rezultati koji se odnose na primenu ILC frakcionog reda gde je prvo PDα tip predložen za linearni sistem frakcionog reda. Pokazano je da se pod odredjenim dovoljnim uslovima koji uključuju operatore učenja konvergencija datog sistema može biti garantovana. Takodje, PIβDα tip ILC upravljanja je predložen za linearni sistem frakcionog reda sa kašnjenjem. Konačno, dovoljni uslovi za konvergenciju u vremenskom domenu predloženog ILC upravljanja su dati odgovarajućom teoremom sa pratećim dokazom.This paper provides an overview of the recently presented and published results relating to the use of iterative learning control (ILC) based on and integer and fractional order. ILC is one of the recent topics in control theories and it is a powerful control concept that iteratively improves the behavior of processes that are repetitive in nature. ILC is suitable for controlling a wider class of mechatronic systems - it is especially suitable for motion control of robotic systems that attract and hold an important position in biomechatronical, technical systems involving the application, military industry, etc. The first part of the paper presents the results relating to the application of higher integer order PD type ILC with numerical simulation. Also, another integer order ILC scheme is proposed for a given robotic system with three degrees of freedom for task-space trajectory tracking where the effectiveness of the suggested control is demonstrated through a simulation procedure. In the second part, the results related to the application of the fractional order of ILC are presented where PDα type of ILC is proposed firstly, for a fractional order linear time invariant system. It is shown that under some sufficient conditions which include the learning operators, convergence of the learning system can be guaranteed. Also, PIβDα type of ILC is suggested for a fractional order linear time delay system. Finally, sufficient conditions for the convergence in the time domain of the proposed ILC were given by the corresponding theorem together with its proof

Fractional Deep Neural Network via Constrained Optimization

This paper introduces a novel algorithmic framework for a deep neural network (DNN), which in a mathematically rigorous manner, allows us to incorporate history (or memory) into the network -- it ensures all layers are connected to one another. This DNN, called Fractional-DNN, can be viewed as a time-discretization of a fractional in time nonlinear ordinary differential equation (ODE). The learning problem then is a minimization problem subject to that fractional ODE as constraints. We emphasize that an analogy between the existing DNN and ODEs, with standard time derivative, is well-known by now. The focus of our work is the Fractional-DNN. Using the Lagrangian approach, we provide a derivation of the backward propagation and the design equations. We test our network on several datasets for classification problems. Fractional-DNN offers various advantages over the existing DNN. The key benefits are a significant improvement to the vanishing gradient issue due to the memory effect, and better handling of nonsmooth data due to the network's ability to approximate non-smooth functions

Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches

Imaging spectrometers measure electromagnetic energy scattered in their instantaneous field view in hundreds or thousands of spectral channels with higher spectral resolution than multispectral cameras. Imaging spectrometers are therefore often referred to as hyperspectral cameras (HSCs). Higher spectral resolution enables material identification via spectroscopic analysis, which facilitates countless applications that require identifying materials in scenarios unsuitable for classical spectroscopic analysis. Due to low spatial resolution of HSCs, microscopic material mixing, and multiple scattering, spectra measured by HSCs are mixtures of spectra of materials in a scene. Thus, accurate estimation requires unmixing. Pixels are assumed to be mixtures of a few materials, called endmembers. Unmixing involves estimating all or some of: the number of endmembers, their spectral signatures, and their abundances at each pixel. Unmixing is a challenging, ill-posed inverse problem because of model inaccuracies, observation noise, environmental conditions, endmember variability, and data set size. Researchers have devised and investigated many models searching for robust, stable, tractable, and accurate unmixing algorithms. This paper presents an overview of unmixing methods from the time of Keshava and Mustard's unmixing tutorial [1] to the present. Mixing models are first discussed. Signal-subspace, geometrical, statistical, sparsity-based, and spatial-contextual unmixing algorithms are described. Mathematical problems and potential solutions are described. Algorithm characteristics are illustrated experimentally.Comment: This work has been accepted for publication in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensin

An Improved Variable Structure Adaptive Filter Design and Analysis for Acoustic Echo Cancellation

In this research an advance variable structure adaptive Multiple Sub-Filters (MSF) based algorithm for single channel Acoustic Echo Cancellation (AEC) is proposed and analyzed. This work suggests a new and improved direction to find the optimum tap-length of adaptive filter employed for AEC. The structure adaptation, supported by a tap-length based weight update approach helps the designed echo canceller to maintain a trade-off between the Mean Square Error (MSE) and time taken to attain the steady state MSE. The work done in this paper focuses on replacing the fixed length sub-filters in existing MSF based AEC algorithms which brings refinements in terms of convergence, steady state error and tracking over the single long filter, different error and common error algorithms. A dynamic structure selective coefficient update approach to reduce the structural and computational cost of adaptive design is discussed in context with the proposed algorithm. Simulated results reveal a comparative performance analysis over proposed variable structure multiple sub-filters designs and existing fixed tap-length sub-filters based acoustic echo cancellers