Symbolic Time Series Analysis in Economics

In this paper I describe and apply the methods of Symbolic Time Series Analysis (STSA) to an experimental framework. The idea behind Symbolic Time Series Analysis is simple: the values of a given time series data are transformed into a finite set of symbols obtaining a finite string. Then, we can process the symbolic sequence using tools from information theory and symbolic dynamics. I discuss data symbolization as a tool for identifying temporal patterns in experimental data and use symbol sequence statistics in a model strategy. To explain these applications, I describe methods to select the symbolization of the data (Section 2), I introduce the symbolic sequence histograms and some tools to characterize and compare these histograms (Section 3). I show that the methods of symbolic time series analysis can be a good tool to describe and recognize time patterns in complex dynamical processes and to extract dynamical information about this kind of system. In particular, the method gives us a language in which to express and analyze these time patterns. In section 4 I report some applications of STSA to study the evolution of ifferent economies. In these applications data symbolization is based on economic criteria using the notion of economic regime introduced earlier in this thesis. I use STSA methods to describe the dynamical behavior of these economies and to do comparative analysis of their regime dynamics. In section 5 I use STSA to reconstruct a model of a dynamical system from measured time series data. In particular, I will show how the observed symbolic sequence statistics can be used as a target for measuring the goodness of fit of proposed models.

Music adapting to the brain: From diffusion chains to neurophysiology

During the last decade, the use of experimental approaches on cultural evolution research has provided novel insights, and supported theoretical predictions, on the principles driving the evolution of human cultural systems. Laboratory simulations of language evolution showed how general-domain constraints on learning, in addition to pressures for language to be expressive, may be responsible for the emergence of linguistic structure. Languages change when culturally transmitted, adapting to fit, among all, the cognitive abilities of their users. As a result, they become regular and compressed, easier to acquire and reproduce. Although a similar theory has been recently extended to the musical domain, the empirical investigation in this field is still scarce. In addition, no study to our knowledge directly addressed the role of cognitive constraints in cultural transmission with neurophysiological investigation. In my thesis I addressed both these issues with a combination of behavioral and neurophysiological methods, in three experimental studies. In study 1 (Chapter 2), I examined the evolution of structural regularities in artificial melodic systems while they were being transmitted across individuals via coordination and alignment. To this purpose I used a new laboratory model of music transmission: the multi-generational signaling games (MGSGs), a variant of the signaling games. This model combines classical aspects of lab-based semiotic models of communication, coordination and interaction (horizontal transmission), with the vertical transmission across generations of the iterated learning model (vertical transmission). Here, two-person signaling games are organized in diffusion chains of several individuals (generations). In each game, the two players (a sender and a receiver) must agree on a common code - here a miniature system where melodic riffs refer to emotions. The receiver in one game becomes the sender in the next game, possibly retransmitting the code previously learned to another generation of participants, and so on to complete the diffusion chain. I observed the gradual evolution of several structures features of musical phrases over generations: proximity, continuity, symmetry, and melodic compression. Crucially, these features are found in most of musical cultures of the world. I argue that we tapped into universal processing mechanisms of structured sequence processing, possibly at work in the evolution of real music. In study 2 (Chapter 3), I explored the link between cultural adaptation and neural information processing. To this purpose, I combined behavioral and EEG study on 2 successive days. I show that the latency of the mismatch negativity (MMN) recorded in a pre-attentive auditory sequence processing task on day 1, predicts how well participants learn and transmit an artificial tone system with affective semantics in two signaling games on day 2. Notably, MMN latencies also predict which structural changes are introduced by participants into the artificial tone system. In study 3 (Chapter 4), I replicated and extended behavioral and neurophysiological findings on the temporal domain of music, with two independent experiments. In the first experiment, I used MGSGs as a laboratory model of cultural evolution of rhythmic equitone patterns referring to distinct emotions. As a result of transmission, rhythms developed a universal property of music structure, namely temporal regularity (or isochronicity). In the second experiment, I anchored this result with neural predictors. I showed that neural information processing capabilities of individuals, as measured with the MMN on day 1, can predict learning, transmission, and regularization of rhythmic patterns in signaling games on day 2. In agreement with study 2, I observe that MMN brain timing may reflect the efficiency of sensory systems to process auditory patterns. Functional differences in those systems, across individuals, may produce a different sensitivity to pressures for regularities in the cultural system. Finally, I argue that neural variability can be an important source of variability of cultural traits in a population. My work is the first to systematically describe the emergence of structural properties of melodic and rhythmic systems in the laboratory, using an explicit game-theoretic model of cultural transmission in which agents freely interact and exchange information. Critically, it provides the first demonstration that social learning, transmission, and cultural adaptation are constrained and driven by individual differences in the functional organization of sensory systems

Unsupervised Similarity-Based Risk Stratification for Cardiovascular Events Using Long-Term Time-Series Data

In medicine, one often bases decisions upon a comparative analysis of patient data. In this paper, we build upon this observation and describe similarity-based algorithms to risk stratify patients for major adverse cardiac events. We evolve the traditional approach of comparing patient data in two ways. First, we propose similarity-based algorithms that compare patients in terms of their long-term physiological monitoring data. Symbolic mismatch identifies functional units in long-term signals and measures changes in the morphology and frequency of these units across patients. Second, we describe similarity-based algorithms that are unsupervised and do not require comparisons to patients with known outcomes for risk stratification. This is achieved by using an anomaly detection framework to identify patients who are unlike other patients in a population and may potentially be at an elevated risk. We demonstrate the potential utility of our approach by showing how symbolic mismatch-based algorithms can be used to classify patients as being at high or low risk of major adverse cardiac events by comparing their long-term electrocardiograms to that of a large population. We describe how symbolic mismatch can be used in three different existing methods: one-class support vector machines, nearest neighbor analysis, and hierarchical clustering. When evaluated on a population of 686 patients with available long-term electrocardiographic data, symbolic mismatch-based comparative approaches were able to identify patients at roughly a two-fold increased risk of major adverse cardiac events in the 90 days following acute coronary syndrome. These results were consistent even after adjusting for other clinical risk variables.National Science Foundation (U.S.) (CAREER award 1054419

Neural Models of Normal and Abnormal Behavior: What Do Schizophrenia, Parkinsonism, Attention Deficit Disorder, and Depression Have in Common?

Defense Advanced Research Projects Agency and Office of Naval Research (N00014-95-1-0409); National Science Foundation (IRI-97-20333

An Overview of the Use of Neural Networks for Data Mining Tasks

In the recent years the area of data mining has experienced a considerable demand for technologies that extract knowledge from large and complex data sources. There is a substantial commercial interest as well as research investigations in the area that aim to develop new and improved approaches for extracting information, relationships, and patterns from datasets. Artificial Neural Networks (NN) are popular biologically inspired intelligent methodologies, whose classification, prediction and pattern recognition capabilities have been utilised successfully in many areas, including science, engineering, medicine, business, banking, telecommunication, and many other fields. This paper highlights from a data mining perspective the implementation of NN, using supervised and unsupervised learning, for pattern recognition, classification, prediction and cluster analysis, and focuses the discussion on their usage in bioinformatics and financial data analysis tasks

Numerical Cognition in Rhesus Monkeys (Macaca mulatta)

Over the past few decades, researchers have firmly established that a wide range of nonhuman animals exhibit some form of numerical competence. The focus of this research was to define further the extent of numerical ability in rhesus monkeys, and specifically to determine whether the animals possess a symbolic understanding of Arabic numerals. This required examining the stimulus attributes (e.g., number vs. hedonic value) represented by the numerals, as well as the precision (e.g., absolute vs. relative) and generality of those representations. In chapters 2 and 3, monkeys were required to compare and order numerals and were rewarded with either proportional or probabilistic rewards. The results indicated that monkeys were relying on the ordinal or absolute numerical values associated with each numeral and not hedonic value or learned 2-choice discriminations. The studies in chapters 4 and 5 indicated that monkeys can use numerals to symbolize an approximate number of sequential motor responses. The study in Chapter 6 tested the generality of the monkeys’ symbolic number concept using transfer tests. The results indicated that some monkeys are able to abstract number across presentation mode, but this ability is only exhibited under limited conditions. Collectively, these studies provide evidence that rhesus monkeys view Arabic numerals as more than sign-stimuli associated with specific response-reward histories, but that numerals do not have the same precise symbolic meaning as they do for humans

Kompozicionalni hierarhični model za pridobivanje informacij iz glasbe

In recent years, deep architectures, most commonly based on neural networks, have advanced the state of the art in many research areas. Due to the popularity and the success of deep neural-networks, other deep architectures, including compositional models, have been put aside from mainstream research. This dissertation presents the compositional hierarchical model as a novel deep architecture for music processing. Our main motivation was to develop and explore an alternative non-neural deep architecture for music processing which would be transparent, meaning that the encoded knowledge would be interpretable, trained in an unsupervised manner and on small datasets, and useful as a feature extractor for classification tasks, as well as a transparent model for unsupervised pattern discovery. We base our work on compositional models, as compositionality is inherent in music. The proposed compositional hierarchical model learns a multi-layer hierarchical representation of the analyzed music signals in an unsupervised manner. It provides transparent insights into the learned concepts and their structure. It can be used as a feature extractor---its output can be used for classification tasks using existing machine learning techniques. Moreover, the model\u27s transparency enables an interpretation of the learned concepts, so the model can be used for analysis (exploration of the learned hierarchy) or discovery-oriented (inferring the hierarchy) tasks, which is difficult with most neural network based architectures. The proposed model uses relative coding of the learned concepts, which eliminates the need for large annotated training datasets that are essential in deep architectures with a large number of parameters. Relative coding contributes to slim models, which are fast to execute and have low memory requirements. The model also incorporates several biologically-inspired mechanisms that are modeled according to the mechanisms that exists at the lower levels of human perception (eg~ lateral inhibition in the human ear) and that significantly affect perception. The proposed model is evaluated on several music information retrieval tasks and its results are compared to the current state of the art. The dissertation is structured as follows. In the first chapter we present the motivation for the development of the new model. In the second chapter we elaborate on the related work in music information retrieval and review other compositional and transparent models. Chapter three introduces a thorough description of the proposed model. The model structure, its learning and inference methods are explained, as well as the incorporated biologically-inspired mechanisms. The model is then applied to several different music domains, which are divided according to the type of input data. In this we follow the timeline of the development and the implementation of the model. In chapter four, we present the model\u27s application to audio recordings, specifically for two tasks: automatic chord estimation and multiple fundamental frequency estimation. In chapter five, we present the model\u27s application to symbolic music representations. We concentrate on pattern discovery, emphasizing the model\u27s ability to tackle such problems. We also evaluate the model as a feature generator for tune family classification. Finally, in chapter six, we show the latest progress in developing the model for representing rhythm and show that it exhibits a high degree of robustness in extracting high-level rhythmic structures from music signals. We conclude the dissertation by summarizing our work and the results, elaborating on forthcoming work in the development of the model and its future applications.S porastom globokih arhitektur, ki temeljijo na nevronskih mrežah, so se v zadnjem času bistveno izboljšali rezultati pri reševanju problemov na več področjih. Zaradi popularnosti in uspešnosti teh globokih pristopov, temelječih na nevronskih mrežah, so bili drugi, predvsem kompozicionalni pristopi, odmaknjeni od središča pozornosti raziskav. V pričujoči disertaciji se posvečamo vprašanju, ali je mogoče razviti globoko arhitekturo, ki bo presegla obstoječe probleme globokih arhitektur. S tem namenom se vračamo h kompozicionalnim modelom in predstavimo kompozicionalni hierarhični model kot alternativno globoko arhitekturo, ki bo imela naslednje značilnosti: transparentnost, ki omogoča enostavno razlago naučenih konceptov, nenadzorovano učenje in zmožnost učenja na majhnih podatkovnih bazah, uporabnost modela kot izluščevalca značilk, kot tudi zmožnost uporabe transparentnosti modela za odkrivanje vzorcev. Naše delo temelji na kompozicionalnih modelih, ki so v glasbi intuitivni. Predlagani kompozicionalni hierarhični model je zmožen nenadzorovanega učenja večnivojske predstavitve glasbenega vhoda. Model omogoča pregled naučenih konceptov skozi transparentne strukture. Lahko ga uporabimo kot generator značilk -- izhod modela lahko uporabimo za klasifikacijo z drugimi pristopi strojnega učenja. Hkrati pa lahko transparentnost predlaganega modela uporabimo za analizo (raziskovanje naučene hierarhije) pri odkrivanju vzorcev, kar je težko izvedljivo z ostalimi pristopi, ki temeljijo na nevronskih mrežah. Relativno kodiranje konceptov v samem modelu pripomore k precej manjšim modelom in posledično zmanjšuje potrebo po velikih podatkovnih zbirkah, potrebnih za učenje modela. Z vpeljavo biološko navdahnjenih mehanizmov želimo model še bolj približati človeškemu načinu zaznave. Za nekatere mehanizme, na primer inhibicijo, vemo, da so v človeški percepciji prisotni na nižjih nivojih v ušesu in bistveno vplivajo na način zaznave. V modelu uvedemo prve korake k takšnemu načinu procesiranja proti končnemu cilju izdelave modela, ki popolnoma odraža človeško percepcijo. V prvem poglavju disertacije predstavimo motivacijo za razvoj novega modela. V drugem poglavju se posvetimo dosedanjim objavljenim dosežkom na tem področju. V nadaljnjih poglavjih se osredotočimo na sam model. Sprva opišemo teoretično zasnovo modela in način učenja ter delovanje biološko-navdahnjenih mehanizmov. V naslednjem koraku model apliciramo na več različnih glasbenih domen, ki so razdeljene glede na tip vhodnih podatkov. Pri tem sledimo časovnici razvoja in implementacijam modela tekom doktorskega študija. Najprej predstavimo aplikacijo modela za časovno-frekvenčne signale, na katerem model preizkusimo za dve opravili: avtomatsko ocenjevanje harmonij in avtomatsko transkripcijo osnovnih frekvenc. V petem poglavju predstavimo drug način aplikacije modela, tokrat na simbolne vhodne podatke, ki predstavljajo glasbeni zapis. Pri tem pristopu se osredotočamo na odkrivanje vzorcev, s čimer poudarimo zmožnost modela za reševanje tovrstnih problemov, ki je ostalim pristopom še nedosegljivo. Model prav tako evalviramo v vlogi generatorja značilk. Pri tem ga evalviramo na problemu melodične podobnosti pesmi in razvrščanja v variantne tipe. Nazadnje, v šestem poglavju, pokažemo zadnji dosežek razvoja modela, ki ga apliciramo na problem razumevanja ritma v glasbi. Prilagojeni model analiziramo in pokažemo njegovo zmožnost učenja različnih ritmičnih oblik in visoko stopnjo robustnosti pri izluščevanju visokonivojskih struktur v ritmu. V zaključkih disertacije povzamemo vloženo delo in rezultate ter nakažemo nadaljnje korake za razvoj modela v prihodnosti