Neural networks for non-linear adaptive filtering

Neural networks are shown to be a class of non-linear adaptive filters, which can be trained permanently with a possibly infinite number of timeordered examples ; this is an altogether différent framework from the usual, non-adaptive training of neural networks . A family of new gradientbased algorithms is proposed.Nous introduisons une famille d'algorithmes adaptatifs permettant l'utilisation de réseaux de neurones comme filtres adaptatifs non linéaires, systèmes susceptibles de subir un apprentissage permanent à partir d'un nombre éventuellement infini d'exemples présentés dans un ordre déterminé. Ces algorithmes, fondés sur des techniques d'évaluation du gradient d'une fonction de coût, s'inscrivent dans un cadre différent de celui de l'apprentissage classique des réseaux de neurones, qui est habituellement non adaptati

Training Recurrent Neural Networks: Why and How ? An Illustration in Dynamical Process Modeling.

The paper first summarizes a general approach to the training of recurrent neural networks by gradient-based algorithms, which leads to the introduction of four families of training algorithms. Because of the variety of possibilities thus available to the "neural network designer", the choice of the appropriate algorithm to solve a given problem becomes critical. We show that, in the case of process modeling, this choice depends on how noise interferes with the process to be modeled; this is evidenced by three examples of modeling of dynamical processes, where the detrimental effect of inappropriate training algorithms on the prediction error made by the network is clearly demonstrated. 1 INTRODUCTION During the past few years, there has been a growing interest in the training of recurrent neural networks, either for associative memory tasks, or for tasks related to grammatical inference, time series prediction, process modeling and process control. A general framework for the trainin..

The selection of neural models of non-linear dynamical systems by statistical tests

Abstract- A procedure for the selection of neural models of dynamical processes is presented. It uses statistical tests at various levels of model reduction, in order to provide optimal tradeoffs between accuracy and parsimony. The efficiency of the method is illustrated by the modeling of a highly non-linear NARX process

Neural Networks for Signal Processing IV, J. Vlontzos, J. Hwang, E. Wilson, eds, pp. 229-237 (IEEE , 1994).

A procedure for the selection of neural models of dynamical processes is presented. It uses statistical tests at various levels of model reduction, in order to provide optimal tradeoffs between accuracy and parsimony. The efficiency of the method is illustrated by the modeling of a highly non-linear NARX process. INTRODUCTION The representation of the behaviour of dynamical processes is a conceptually straightforward application of neural networks, whether feedforward or recurrent, as non-linear regressors. In practice, however, the modeling of a process requires solving several problems: (i) the choice of the nature of the model (static model vs dynamic model, input-output representation vs state representation, ...) requires an analysis of the future use of the model (for instance, whether it will be used for predicting the future evolution of the process, or whether it will be used within a control system), and an analysis of the a priori knowledge on the phenomena involved in the ..

Adaptive Training Of Feedback Neural Networks For Non-Linear Filtering

. The paper proposes a general framework which encompasses the training of neural networks and the adaptation of filters. It is shown that neural networks can be considered as general non-linear filters which can be trained adaptively, i.e. which can undergo continual training. A unified view of gradient-based training algorithms for feedback networks is proposed, which gives rise to new algorithms. The use of some of these algorithms is illustrated by examples of non-linear adaptive filtering and process identification. INTRODUCTION In recent papers [1, 2], a general framework, which encompasses algorithms used for the training of neural networks and algorithms used for the adaptation of filters, has been proposed. Specifically, it was shown that neural networks can be used adaptively, i.e. can undergo continual training with a possibly infinite number of time-ordered examples - in contradistinction to the traditional training of neural networks with a finite number of examples pres..

Diabetes Care

OBJECTIVE We evaluated the association between diabetic retinopathy stages and lower-extremity arterial disease (LEAD), its prognostic value, and the influence of potential contributors to this relationship in a prospective cohort of patients with type 2 diabetes. RESEARCH DESIGN AND METHODS Diabetic retinopathy was staged at baseline as absent, nonproliferative, or proliferative. A Cox regression model was fitted in order to compute the hazard ratio (HR) (95% CI) for major LEAD (lower-limb amputation or revascularization) during follow-up by baseline retinopathy stages. The retinopathy-LEAD association was assessed in subgroups by age, sex, diabetes duration, HbA1c, systolic blood pressure, diabetic kidney disease, smoking, and macrovascular disease at baseline. The performance of retinopathy in stratifying LEAD risk was assessed by using the C statistic, integrated discrimination improvement (IDI), and net reclassification improvement (NRI). RESULTS Among 1,320 participants without a history of LEAD at baseline, 94 (7.1%) developed a major LEAD during a 7.1-year median follow-up (incidence rate 9.6 per 1,000 person-years [95% CI 7.8–11.7]). The LEAD incidence rate (per 1,000 person-years) increased as retinopathy worsened: it was 5.5 (95% CI 3.9–7.8) in participants in whom retinopathy was absent, 14.6 (11.1–19.3) in those with nonproliferative retinopathy, and 20.1 (11.1–36.3) in those with proliferative retinopathy. Nonproliferative retinopathy (adjusted HR 2.31 [95% CI 1.43–3.81], P = 0.0006) and proliferative retinopathy (3.14 [1.40–6.15], P = 0.007) remained associated with major LEAD. No heterogeneity was observed across subgroups. Retinopathy enhanced the C statistic (+0.023 [95% CI 0.003–0.044], P = 0.02), IDI (0.209 [0.130–0.321], P < 0.001), and NRI (0.562 [0.382–0.799], P < 0.001) values for risk of LEAD, beyond traditional risk factors. CONCLUSIONS An independent dose-response relationship was identified between diabetic retinopathy stages and major LEAD. Retinopathy yielded incremental prognostic information for stratifying risk of LEAD, suggesting its usefulness as a predictor of LEAD

Diabetes Care

OBJECTIVE: Inflammation and oxidative stress play an important role in the pathogenesis of lower-extremity artery disease (LEAD). We assessed the prognostic values of inflammatory and redox status biomarkers on the risk of LEAD in individuals with type 2 diabetes. RESEARCH DESIGN AND METHODS: Plasma concentrations of tumor necrosis factor-alpha receptor 1 (TNFR1), angiopoietin-like 2, ischemia-modified albumin (IMA), fluorescent advanced glycation end products, protein carbonyls, and total reductive capacity of plasma were measured at baseline in the SURDIAGENE (Survie, Diabete de type 2 et Genetique) cohort. Major LEAD was defined as the occurrence during follow-up of peripheral revascularization or lower-limb amputation. RESULTS: Among 1,412 participants at baseline (men 58.2%, mean [SD] age 64.7 [10.6] years), 112 (7.9%) developed major LEAD during 5.6 years of follow-up. High plasma concentrations of TNFR1 (HR [95% CI] for second vs. first tertile 1.12 [0.62-2.03; P = 0.71], third vs. first tertile 2.16 [1.19-3.92; P = 0.01]) and IMA (2.42 [1.38-4.23; P = 0.002], 2.04 [1.17-3.57; P = 0.01]) were independently associated with an increased risk of major LEAD. Plasma concentrations of TNFR1 but not IMA yielded incremental information, over traditional risk factors, for the risk of major LEAD as follows: C-statistic change (0.036 [95% CI 0.013-0.059]; P = 0.002), integrated discrimination improvement (0.012 [0.005-0.022]; P < 0.001), continuous net reclassification improvement (NRI) (0.583 [0.294-0.847]; P < 0.001), and categorical NRI (0.171 [0.027-0.317]; P = 0.02). CONCLUSIONS: Independent associations exist between high plasma TNFR1 or IMA concentrations and increased 5.6-year risk of major LEAD in people with type 2 diabetes. TNFR1 allows incremental prognostic information, suggesting its use as a biomarker for LEAD