Statistical Complexity and Nontrivial Collective Behavior in Electroencephalografic Signals

We calculate a measure of statistical complexity from the global dynamics of electroencephalographic (EEG) signals from healthy subjects and epileptic patients, and are able to stablish a criterion to characterize the collective behavior in both groups of individuals. It is found that the collective dynamics of EEG signals possess relative higher values of complexity for healthy subjects in comparison to that for epileptic patients. To interpret these results, we propose a model of a network of coupled chaotic maps where we calculate the complexity as a function of a parameter and relate this measure with the emergence of nontrivial collective behavior in the system. Our results show that the presence of nontrivial collective behavior is associated to high values of complexity; thus suggesting that similar dynamical collective process may take place in the human brain. Our findings also suggest that epilepsy is a degenerative illness related to the loss of complexity in the brain.Comment: 13 pages, 3 figure

Minimal approach to neuro-inspired information processing

© 2015 Soriano, Brunner, Escalona-Morán, Mirasso and Fischer. To learn and mimic how the brain processes information has been a major research challenge for decades. Despite the efforts, little is known on how we encode, maintain and retrieve information. One of the hypothesis assumes that transient states are generated in our intricate network of neurons when the brain is stimulated by a sensory input. Based on this idea, powerful computational schemes have been developed. These schemes, known as machine-learning techniques, include artificial neural networks, support vector machine and reservoir computing, among others. In this paper, we concentrate on the reservoir computing (RC) technique using delay-coupled systems. Unlike traditional RC, where the information is processed in large recurrent networks of interconnected artificial neurons, we choose a minimal design, implemented via a simple nonlinear dynamical system subject to a self-feedback loop with delay. This design is not intended to represent an actual brain circuit, but aims at finding the minimum ingredients that allow developing an efficient information processor. This simple scheme not only allows us to address fundamental questions but also permits simple hardware implementations. By reducing the neuro-inspired reservoir computing approach to its bare essentials, we find that nonlinear transient responses of the simple dynamical system enable the processing of information with excellent performance and at unprecedented speed. We specifically explore different hardware implementations and, by that, we learn about the role of nonlinearity, noise, system responses, connectivity structure, and the quality of projection onto the required high-dimensional state space. Besides the relevance for the understanding of basic mechanisms, this scheme opens direct technological opportunities that could not be addressed with previous approaches.The authors acknowledge support by MINECO (Spain) under Projects TEC2012-36335 (TRIPHOP) and FIS2012-30634 (Intense@cosyp), FEDER and Govern de les Illes Balears via the program Grups Competitius. The work of MS was supported by the Conselleria d'Educació, Cultura i Universitats del Govern de les Illes Balears and the European Social Fund.Peer Reviewe

Complexity and Information: Measuring Emergence, Self-organization, and Homeostasis at Multiple Scales

Concepts used in the scientific study of complex systems have become so widespread that their use and abuse has led to ambiguity and confusion in their meaning. In this paper we use information theory to provide abstract and concise measures of complexity, emergence, self-organization, and homeostasis. The purpose is to clarify the meaning of these concepts with the aid of the proposed formal measures. In a simplified version of the measures (focusing on the information produced by a system), emergence becomes the opposite of self-organization, while complexity represents their balance. Homeostasis can be seen as a measure of the stability of the system. We use computational experiments on random Boolean networks and elementary cellular automata to illustrate our measures at multiple scales.Comment: 42 pages, 11 figures, 2 table

Computational Properties of Delay-Coupled Systems

Tesis Doctoral presentada por Miguel Angel Escalona Morán para optar al título de Doctor, en el Programa de Física del Departamento de Física de la Universitat de les Illes Balears, realizada en el IFISC bajo la dirección de Claudio Mirasso, catedrático de universidad y Miguel Cornelles Soriano, contratado postdoctoral CAIB.In this research work we study the computational properties of delay-coupled systems. In particular, we use a machine learning technique known as reservoir computing. In machine learning, a computer learns to solve different tasks using examples and without knowing explicitly their solution. For the study of the computational properties, a numerical toolbox, written in Python, was developed. This toolbox allows a fast implementation of the different scenarios described in this thesis. Using a reservoir computer, we studied several computational properties, focusing on its kernel quality, its ability to separate different input samples and the intrinsic memory capacity. This intrinsic memory is related to the delayed- feedback of the reservoir. We used a delay-coupled system as reservoir to study its computational ability in three different kinds of tasks: system’s modeling, time-series prediction and classification tasks. The system’s modeling task was performed using the Nonlinear Autoregressive Moving Average (of ten steps), NARMA10. The NARMA10 model creates autoregressive time series from a set of normally distributed random sequences. The reservoir computer learns how to emulate the system using only the sequence of random numbers and the autoregressive time series, but without knowing the equations of the NARMA10. The results of our approach are equivalent to those published by other authors and show the computational power of our method. For the time-series prediction tasks, we used three kinds of time series: a model that gives the variations in temperature of the sea surface that provoke El Niño phenomenon, the Lorenz system and the dynamics of a chaotic laser. Different scenarios were explored depending on the nature of the time series. For the prediction of the variation in temperature of the sea surface, we perform estimations of one, three and six months in advance. The error was measured as the Normalized Root Mean Square Error (NRMSE). For the different prediction horizons, we obtained errors of 2%, 8% and 24%, respectively. The classification tasks were carried out for a Spoken Digit Recognition (SDR) task and a real-world biomedical task. The SDR was used to illustrate different scenarios of a machine learning problem. The biomedical task consists on the automatic classification of heartbeats with cardiac arrhythmias. We use the MIT-BIH Arrhythmia database, a widely used database in cardiology. For comparison purposes, we followed the guidelines of the Association for the Advancement of Medical Instrumentation for the evaluation of arrhythmia-detector algorithms. We used a biostatistical learning process named logistic regression that allowed to compute the probability that a heartbeat belongs to a particular class.This is in contrast to the commonly used linear regression. The results obtained in this work show the versatility and efficiency of our implemented reservoir computer. Our results are equivalent and show improvement over other reported results on this problem under similar conditions and using the same database. To enhance the computational ability of our delay-coupled system, we included a multivariate scheme that allows the consideration of different variables of a system. We evaluated the influence of this multivariate scenario using a time- series prediction and the classification of heartbeat tasks. The results show improvement in the performance of the reservoir computer in comparison with the same tasks in the univariate case.Peer reviewe

Información Investigador: Escalona Morán, Miguel Ángel

Resumen Curricular Miguel A. Escalona Morán es Licenciado en Física, graduado en 2004 por la Universidad de Los Andes (ULA). En Julio del 2006 recibe el título de Magister Scientiae en Física Fundamental de la ULA. Actualmente lleva a cabo dos programas de doctorado, el de Ciencias Aplicadas de la Facultad de Ingeniería y el de Tratamiento de Señales y Telecomunicaciones de la Universidad de Rennes 1, Francia. Posee conocimientos sobre Análisis No-Lineal de series de tiempo, tales como sincronización caótica, medidas de correlación, etc. Ha trabajado en diversos grupos de investigación de diversas áreas, tales como la Economía (BCV), la Física (CFF), la computación (CeCalCULA), la Medicina (GIBULA, Instituto HelmtHoltz, Lab. de Tratamiento de Señales e Imágenes).Universitario18 - 2005Caos y Sistemas Complejos, Dinámica No Lineal y Análisis de series de tiempo, Física Computacional, Ingeniería Biomédica, Modelado y Simulación.Octubre de 2007Licenciado en Física+58 274 2401330Facultad de Ciencias.Personal de [email protected], [email protected]

Computational properties of delay-coupled systems

[eng] In this research work we study the computational properties of delay-coupled systems. In particular, we use a machine learning technique known as reservoir computing. In machine learning, a computer learns to solve di↵erent tasks using examples and without knowing explicitly their solution. For the study of the computational properties, a numerical toolbox, written in Python, was developed. This toolbox allows a fast implementation of the di↵erent scenarios described in this thesis. Using a reservoir computer, we studied several computational properties, focusing on its kernel quality, its ability to separate di↵erent input samples and the intrinsic memory capacity. This intrinsic memory is related to the delayedfeedback of the reservoir. We used a delay-coupled system as reservoir to study its computational ability in three di↵erent kinds of tasks: system’s modeling, time-series prediction and classification tasks. The system’s modeling task was performed using the Nonlinear Autoregressive Moving Average (of ten steps), NARMA10. The NARMA10 model creates autoregressive time series from a set of normally distributed random sequences. The reservoir computer learns how to emulate the system using only the sequence of random numbers and the autoregressive time series, but without knowing the equations of the NARMA10. The results of our approach are equivalent to those published by other authors and show the computational power of our method. For the time-series prediction tasks, we used three kinds of time series: a model that gives the variations in temperature of the sea surface that provoke El Niño phenomenon, the Lorenz system and the dynamics of a chaotic laser. Di↵erent scenarios were explored depending on the nature of the time series. For the prediction of the variation in temperature of the sea surface, we perform estimations of one, three and six months in advance. The error was measured as the Normalized Root Mean Square Error (NRMSE). For the di↵erent prediction horizons, we obtained errors of 2%, 8% and 24%, respectively. The classification tasks were carried out for a Spoken Digit Recognition (SDR) task and a real-world biomedical task. The SDR was used to illustrate different scenarios of a machine learning problem. The biomedical task consists on the automatic classification of heartbeats with cardiac arrhythmias. We use the MIT-BIH Arrhythmia database, a widely used database in cardiology. For comparison purposes, we followed the guidelines of the Association for the Advancement of Medical Instrumentation for the evaluation of arrhythmia-detector algorithms. We used a biostatistical learning process named logistic regression that allowed to compute the probability that a heartbeat belongs to a particular class. This is in contrast to the commonly used linear regression. The results obtained in this work show the versatility and efficiency of our implemented reservoir computer. Our results are equivalent and show improvement over other reported results on this problem under similar conditions and using the same database. To enhance the computational ability of our delay-coupled system, we included a multivariate scheme that allows the consideration of di↵erent variables of a system. We evaluated the influence of this multivariate scenario using a timeseries prediction and the classification of heartbeat tasks. The results show improvement in the performance of the reservoir computer in comparison with the same tasks in the univariate case.[spa] En esta tesis se estudian las propiedades computacionales de sistemas acoplados con retraso. En particular la técnica de machine learning, conocida como reservoir computing, es utilizada. En esta técnica el ordenador aprende a resolver tareas a partir de ejemplos que se han dado previamente pero sin indicarle de forma explícita la forma de resolver estos problemas. El desarrollo de este trabajo incluye la creación de una herramienta computacional, escrita en lenguaje Python para la ejecución de los diferentes escenarios presentados en este trabajo. Con la implementación de un sistema acoplado con retraso, hemos estudiado las propiedades de cómputo de este tipo de sistemas, interesándonos principalmente en la calidad del sistema acoplado, su habilidad de separación de elementos distintos y su capacidad intrínseca de memoria, la cual está asociada a la presencia de una retroalimentación retrasada. El sistema se ha usado para demostrar el poder de cálculo que ofrecen los sistemas acoplados con retraso. Se utilizaron tres tipos de tareas: modelado, predicción de series de tiempo y clasificación. El modelado se realizó utilizando el sistema Nonlinear Autoregressive Moving Average de 10 pasos (NARMA10). Este sistema, construye series temporales autoregresivas a partir de series de números aleatorios. El ordenador basado en reservoir aprende a emular este sistema (sin conocer de forma explícita las ecuaciones del mismo) a partir de las secuencias de números aleatorios y las series temporales autoregresivas. Los resultados obtenidos son equivalentes a los publicados por otros autores, demostrando el poder computacional de este método. Para la predicción de series temporales se usaron modelos de variación de temperatura que provocan la aparición del fenómeno de El Niño, el sistema de Lorenz en régimen caótico y la dinámica de un laser caótico. Las estimaciones de series temporales se realizaron bajo diversas circunstancias dependiendo de la naturaleza de las series. Para el caso de El Niño, se realizaron predicciones a uno, tres y seis meses con errores de estimación, determinados por el Normalized Root Mean Square Error (NRMSE) de 2%, 8% y 24%, respectivamente. Como primera tarea de clasificación, se utilizó la tarea de Spoken Digit Recognition y se utilizó para ilustrar diferentes escenarios posibles de un sistema acoplado con retraso. La segunda tarea de clasificación y la mas exhaustiva, se realizó en un problema real de origen biomédico: la clasificación de latidos cardiacos para el caso de arritmias. Se utilizo la base de datos MIT-BIH Arrhythmia la cual ha sido ampliamente usada en cardiología. Por motivos de comparación de resultados, se siguieron las recomendaciones dadas por la Association for the Advancement of Medical Instrumentation para la evaluación de algoritmos detectores de arritmias. Se utilizo un método de entrenamiento del reservoir computer llamado regresión logística en lugar del comúnmente usado: la regresión lineal. La regresión logística nos permite obtener como resultado la probabilidad de que un latido cardiaco pertenezca a una clase u a otra. Los resultados obtenidos demuestran la versatilidad y eficacia de nuestro método de calculo, ya que son equivalentes e incluso mejores a los resultados publicados por otros trabajos bajo circunstancias similares de evaluación y utilizando la misma base de datos. Para mejorar la capacidad de computación del sistema con retraso, se incluyeron variables dinámicas adicionales en nuestro sistema para evaluar el efecto en la predicción de series de tiempo y la clasificación de latidos cardíacos. Los resultados mostraron una mejora sustancial en comparación con el caso en que sólo una variable o canal del electrocardiograma fue usado para realizar la tarea dada.[cat] En aquesta tesi s’estudien les propietats computacionals de sistemes acoblats ambretard. En particular, hemutilitzat la tècnica de "machine learning" coneguda com reservoir computing. En aquesta tècnica, l’ordinador aprèn a resoldre tasques a partir d’exemples que s’han donat prèviament però sense indicar-li de forma explícita la forma de resoldre aquests problemes. El desenvolupament d’aquest treball inclou la creació d’una eina computacional, escrita en llenguatge Python per a l’execució dels diferents escenaris presentats en aquest treball. Amb la implementació d’un sistema acoblat amb retard, hem estudiat les propietats de còmput d’aquest tipus de sistemes, interessant-nos principalment en la qualitat del sistema acoblat, la seva habilitat de separació d’elements diferents i la seva capacitat intrínseca de memòria, la qual està associada a la presència d’una retroalimentació retardada. El sistema s’ha fet servir per demostrar el poder de càlcul que ofereixen els sistemes acoblats amb retard. Es van utilitzar tres tipus de tasques: modelatge, predicció de sèries de temps i classificació. El modelatge es va realitzar utilitzant el model "Nonlinear Autoregressive Moving Average" de 10 passos (NARMA10). Aquest model, construeix sèries temporals autoregresivas a partir de sèries de nombres aleatoris. L’ordinador basat en "reservoir computing" aprèn a emular aquest model (sense conèixer de forma explícita les equacions del mateix) a partir de les seqüències de nombres aleatoris i les sèries temporals autoregresivas. Els resultats obtinguts són equivalents als publicats per altres autors, demostrant el poder computacional d’aquest mètode. Per a la predicció de sèries temporals es van usar models de variació de temperatura que provoquen l’aparició del fenomen de El Niño, el sistema de Lorenz en règim caòtic i la dinàmica d’un làser caòtic. Les estimacions de sèries temporals es van realitzar sota diverses circumstàncies depenent de la naturalesa de les sèries. Per al cas d’El Niño, es van realitzar prediccions a un, tres i sis mesos amb errors d’estimació, determinats pel "Normalized Root Mean Square Error" (NRMSE) de 2%, 8% i 24%, respectivament. Com a primera tasca de classificació, es va utilitzar la tasca de "Spoken Digit Recognition" i s’han il·lustrat diferents configuracions possibles d’un sistema acoblat amb retard. La segona tasca de classificació i la més exhaustiva, es va realitzar en un problema real d’origen biomèdic: la classificació de batecs cardíacs per al cas d’arítmies. Es va utilitzar la base de dades "MIT-BIH Arrhythmia", la qual ha estat àmpliament usada en cardiologia. Per motius de comparació de resultats, es van seguir les recomanacions donades per la "Association for the Advancement of Medical Instrumentation" per a l’avaluació d’algoritmes detectors d’arítmies. Es va utilitzar un mètode d’entrenament del reservoir computer anomenat regressió logística en lloc del comunament usat: la regressió lineal. La regressió logística ens permet obtenir com a resultat la probabilitat que un batec cardíac pertanyi a una classe o a una altra. Els resultats obtinguts demostren la versatilitat i eficàcia del nostre mètode de càlcul, ja que són equivalents i fins i tot millors als resultats publicats per altres treballs sota circumstàncies similars d’avaluació i fent servir la mateixa base de dades. Per millorar la capacitat de computació del sistema amb retard, es van incloure variables dinàmiques addicionals en el nostre sistema per avaluar l’efecte en la predicció de sèries de temps i la classificació de batecs cardíacs. Els resultats van mostrar una millora substancial en comparació amb el cas en que només una variable o canal de l’electrocardiograma va ser usat per realitzar la tasca donada

Information Processing Using Transient Dynamics of Semiconductor Lasers Subject to Delayed Feedback

The increasing amount of data being generated in different areas of science and technology require novel and efficient techniques of processing, going beyond traditional concepts. In this paper, we numerically study the information processing capabilities of semiconductor lasers subject to delayed optical feedback. Based on the recent concept of reservoir computing, we show that certain tasks, which are inherently hard for traditional computers, can be efficiently tackled by such systems. Major advantages of this approach comprise the possibility of simple and low-cost hardware implementation of the reservoir and ultrafast processing speed. Experimental results corroborate the numerical predictions. © 1995-2012 IEEE.Peer Reviewe

Electrocardiogram Classification Using Reservoir Computing With Logistic Regression

An adapted state-of-the-art method of processing information known as Reservoir Computing is used to show its utility on the open and time-consuming problem of heartbeat classification. The MIT-BIH arrhythmia database is used following the guidelines of the Association for the Advancement of Medical Instrumentation. Our approach requires a computationally inexpensive preprocessing of the electrocardiographic signal leading to a fast algorithm and approaching a real-time classification solution. Our multiclass classification results indicate an average specificity of 97.75% with an average accuracy of 98.43%. Sensitivity and positive predicted value show an average of 84.83% and 88.75%, respectively, what makes our approach significant for its use in a clinical context.This work was supported by the grant FIS2012-30634 (Intense@cosyp) from MINECO (Spain) and FEDER and Grups Competitius, Comunitat Autonoma de les Illes Balears, Spain.Peer Reviewe