Transient Information Flow in a Network of Excitatory and Inhibitory Model Neurons: Role of Noise and Signal Autocorrelation

We investigate the performance of sparsely-connected networks of integrate-and-fire neurons for ultra-short term information processing. We exploit the fact that the population activity of networks with balanced excitation and inhibition can switch from an oscillatory firing regime to a state of asynchronous irregular firing or quiescence depending on the rate of external background spikes. We find that in terms of information buffering the network performs best for a moderate, non-zero, amount of noise. Analogous to the phenomenon of stochastic resonance the performance decreases for higher and lower noise levels. The optimal amount of noise corresponds to the transition zone between a quiescent state and a regime of stochastic dynamics. This provides a potential explanation on the role of non-oscillatory population activity in a simplified model of cortical micro-circuits.Comment: 27 pages, 7 figures, to appear in J. Physiology (Paris) Vol. 9

Sports Analytics: Predicting Athletic Performance with a Genetic Algorithm

Existing predictive modeling in sports analytics often hinges on atheoretical assumptions winnowed from a large and diverse pool of game metrics. Feature subset selection by way of a genetic algorithm to identify and assess the combinatorial advantage for a group of metrics is a viable option to otherwise arbitrary model construction. However, this approach concedes similar arbitrariness as there is no general strategy or common practice design among the tightly coupled nucleus of genetic operators. The resulting dizzying ecosystem of choice is especially difficult to overcome and leaves a residual uncertainty regarding true strength of output, specifically for practical implementations. This study transposes ideas from extreme environmental change into a quasi-deterministic extension of standard GA functionality that seeks to punctuate converged populations with individuals from auxiliary metas. This strategy has the effect of challenging what might otherwise be considered shallow fitness, thereby promoting greater trust in output against innumerable alternatives

Neurocognitive Informatics Manifesto.

Informatics studies all aspects of the structure of natural and artificial information systems. Theoretical and abstract approaches to information have made great advances, but human information processing is still unmatched in many areas, including information management, representation and understanding. Neurocognitive informatics is a new, emerging field that should help to improve the matching of artificial and natural systems, and inspire better computational algorithms to solve problems that are still beyond the reach of machines. In this position paper examples of neurocognitive inspirations and promising directions in this area are given

결합된 진동자들의 동기화에 대한 유효 포텐셜 및 기계학습 접근법

학위논문 (박사) -- 서울대학교 대학원 : 자연과학대학 물리학과, 2021. 2. 강병남.Systems with multiple interacting elements exhibit collective behaviors. As one of examples of collective behaviors, synchronization is a process of coordinating two or more elements to realize the system in unison. It is an omnipresent phenomena in nature, for instance, firefly flashing, cricket chirping, cardiac pacemaker cell, and so on. To understand and describe the mechanism of synchronization phenomena, coupled oscillator system is often adopted as the most conventional and suitable model for interacting system. Each oscillator has own frequency representing each unique characteristics, and its phase is adjusted through the interaction with other oscillators on the system. On the way to phase synchronization, such interactions or connections between oscillators can be expressed as links on the complex network and each element (oscillator) is then denoted by a node. A number of studies for coupled oscillators on complex networks have been progressed over the past two decades. Among the studies for synchronization of coupled oscillator systems, the Kuramoto model has played a crucial role as a simple and representative model for describing such collective behavior. Owing to its rich properties such as chaotic dynamical behavior and synchronization transition, the Kuramoto model is an appropriate model to explore. First, fundamental results of previous studies on synchronization of the coupled oscillator system, especially the Kuramoto model, are introduced. This dissertation is composed of two main studies for the coupled oscillator system by adopting two different approaches, respectively. As the first main study of this dissertation, we examine the Kuramoto model using analytical way, the effective potential approach. The Kuramoto model exhibits different types of synchronization transitions depending on the type of natural frequency distribution. To obtain these results, the Kuramoto self-consistency equation (SCE) approach has been used successfully. However, this approach affords only limited understanding of more detailed properties such as the stability. We here extend the SCE approach by introducing an effective potential, that is, an integral version of the SCE. We examine the landscape of this effective potential for second-order, first-order, and hybrid synchronization transitions in the thermodynamic limit. In particular, for the hybrid transition, we find that the minimum of effective potential displays a plateau across the region in which the order parameter jumps. This result suggests that the effective potential can be used to determine a type of synchronization transition. In the second study for the coupled oscillator systems, we applied the machine learning approach to investigate the system based on data-driven analysis and to figure out whether the methodology can be extended to the real world system. With growing interest in the machine learning, recent works on physical systems has demonstrated successful progresses by adopting the machine learning approaches for tasks of classification and generation. We here perform various machine learning approaches to the Kuramoto system which is basic model for synchronization phenomena and exhibits complicated chaotic behavior. As the system displays rich properties such as synchronization transition and nonlinearity with varying parameters, we applied machine learning for finding the value of the coupling strength and the critical value. Considering the finite size scaling, we confirm that results follow the critical behavior of the Kuramoto system. By focusing on the phase dynamics of all oscillators, we applied the performance of the artificial neural network for predicting future behaviors of all oscillators and detecting underlying real brain network topology. As the Kuramoto model offers support for the application on real-world systems exhibiting synchronization phenomena or nonlinear behaviors, our work has potential for utilizing the machine learning approaches to such systems.서로 간의 상호작용이 있는 다수의 개체로 구성된 계는 집단적인 행동을 보인다는 것이 잘 알려져있다. 그러한 집단적인 행동의 대표적인 예로써, 동기화 현상은 두 개 이상의 개체가 상호작용을 통해 모두 동일한 상태에 이르게 되는 과정을 뜻한다. 반딧불의 깜빡임, 귀뚜라미의 울음소리, 심장박동원세포 등 자연에는 동기화 현상의 수많은 예들이 있다. 동기화 현상을 이해하고 묘사하기 위한 가장 대표적이고 적합한 모형으로, 결합된 진동자들로 이루어진 시스템을 생각해 볼 수 있다. 시스템에 있는 각각의 진동자들은 각자의 특성을 나타내는 고유 진동수(natural frequency)를 갖고 있으며, 각각의 위상(phase)들은 시스템의 다른 진동자들과의 상호작용을 통해 시간이 지남에 따라 점차 맞추어 나가게 된다. 이 때, 이러한 위상 동기화가 일어나는 과정에서 진동자들 사이의 연결 또는 상호작용들은 복잡계 네트워크 위의 링크(link)로 표현될 수 있으며, 각각의 개체 혹은 진동자들은 노드(node)로 표현된다. 이러한 결합된 진동자들에 대한 수많은 연구들이 지난 20여년간 이루어져 왔다. 집단현상을 묘사하는 간단하면서도 대표적인 모형인 구라모토 모형을 차용하여 결합된 진동자들의 동기화 현상에 대한 많은 연구들이 진행되어왔다. 구라모토 모형은 카오스 동역학, 동기화 상전이 등의 다양한 특성을 나타내는만큼, 흥미로운 연구들이 많이 이루어져 왔는데, 먼저, 구라모토 모형에서 나타나는 동기화 현상에 대한 선행연구들에서 밝혀진 중요한 결과 및 배경들을 이 학위 논문의 앞부분에서 소개하였다. 그리고 각각을 주요한 연구주제로써, 결합된 진동자들의 시스템에 대한 두 가지 방법론을 사용하여 논문을 구성을 하였다. 첫번째 연구에서는, 유효 포텐셜(effective potential)을 이용한 방법론을 도입하여 해석적인 방법으로 구라모토 모형을 분석하였다. 구라모토 모형에서는 고유 진동수의 분포형태가 변함에 따라 동기화 상전이의 유형또한 변하게 되는데, 구라모토 방정식으로부터 유도한 자기일관성 방정식(self-consistency equation)을 사용하여 이러한 결과를 해석적으로 분석할 수 있다. 하지만, 이러한 방법은 시스템의 안정성과 같은 상세한 특징을 파악하는 데에는 어려움이 있다. 이 연구에서는, 자기일관성 방정식을 적분하여 유도한 유효포텐셜 방법론을 도입하여, 열역학적 극한에 있는 시스템에 대하여 1차 상전이, 2차 상전이 뿐만 아니라 하이브리드 동기화 상전이가 나타날 때의 포텐셜 경관(potential landscape)을 파악하였으며, 특히, 하이브리드 상전이에서는 유효 포텐셜의 최솟값이 임계점에서 평평한 형태를 보인다는 것을 확인하였다. 이러한 결과들은 동기화 상전이의 형태를 파악하는 데에 있어서 유효 포텐셜이 주요한 역할을 해줄 수 있음을 의미한다. 두번째 연구에서는, 데이터 기반 방법론인 기계학습을 사용하여 결합된 진동자들의 시스템을 파악하고, 이러한 방법이 실제의 시스템에 대해서도 확장이 가능한지에 대하여 연구하였다. 최근, 과학분야 뿐만아니라 여러 다양한 분야에서 기계학습에 대한 관심이 높아져 왔는데, 물리적 계에 대해서도 기계학습을 이용한 분류 및 생성 작업을 통해 많은 발전이 이루어져왔다. 본 연구에서는, 여러 기계학습의 모형들을 이용해 구라모토 모형에서 보이는 동기화 상전이 및 비선형, 카오스 동역학을 분석하였다. 질서변수의 시간에 따른 동역학으로부터 진동자들 사이에 내재된 상호작용을 찾고, 진동자들의 위상으로 부터 동기화된 상태와 비동기화된 상태를 구분하여 임계점을 찾는 데에 기계학습 방법을 적용시켜 보았다. 유한 크기 축적 방법(finite-size scaling)을 이용하여 이러한 결과들이 기존의 알려진 구라모토 모형에 대한 눈금 바꿈 행태(scaling behavior)의 결과와 일관성이 있는 것을 확인하였다. 또한, 모든 진동자들의 위상 동역학을 인공 신경망에 입력으로 넣어줌으로써, 진동자들의 이후의 동역학 행태를 파악할 뿐만 아니라, 기저에 깔려 있는 실제 쥐의 시각 피질 네트워크를 알아내는 연구를 진행하였다. 따라서, 동기화 현상 및 비선형 동역학을 보이는 여러 실제의 시스템들에 대한 구라모토 모형의 응용이 가능함에 따라, 본 연구는 그러한 시스템에 대해서도 기계학습 방법을 활용할 수 있는 가능성을 내포한다.1 Introduction 1 1.1 Complex network 1 1.2 Coupled oscillators on complex networks 2 1.3 Machine learning 3 2 Synchronization of coupled oscillators 6 2.1 Synchronization 6 2.2 Coupled oscillators 7 2.3 The Kuramoto model 8 2.4 Natural frequency 9 2.4.1 Gaussian distribution 9 2.4.2 Lorentzian distribution 10 2.4.3 Uniform distribution 10 2.5 Sampling of natural frequency 10 2.5.1 Random sampling 11 2.5.2 Regular sampling 11 2.6 Order parameter 13 2.7 Phase transition 14 2.7.1 Synchronization transition 14 2.7.2 Hybrid phase transition 15 2.7.3 Type of synchronization transition 16 2.8 Finite-size scaling 18 2.8.1 Critical exponents 19 2.8.2 Finite-size effect 20 3 Effective potential approach to synchronization transition 26 3.1 Analytic approaches to the Kuramoto model 29 3.1.1 Self-consistency analysis 29 3.1.2 Ott-Antonsen ansatz 31 3.2 Ad hoc free energy 33 3.3 Second-order synchronization transition 37 3.4 First-order synchronization transition 38 3.4.1 Degree-frequency correlation on scale-free network with 2 < λ < 3 38 3.4.2 Dependence of interaction strength on the frequency 42 3.5 Hybrid synchronization transition 45 3.5.1 Uniform distribution g(ω) 45 3.5.2 Degree-frequency correlation on scale-free networks with λ = 3 49 3.5.3 Flat distribution with exponential tails 50 3.5.4 Flat distribution with power-law tails 51 3.6 Summary 55 4 Machine learning approaches to coupled oscillators 56 4.1 Machine learning models 57 4.1.1 Feed-forward neural network 58 4.1.2 Fully-connected neural network 59 4.1.3 Convolutional neural network 59 4.1.4 Recurrent neural network 59 4.1.5 Reservoir computing 59 4.2 Supervised learning 61 4.3 Finding the coupling strength 62 4.4 Finding the synchronized state 65 4.5 Application I : Prediction of the phase dynamics 68 4.6 Application II : Reconstruction of the network structure 72 4.7 Summary 74 5 Conclusion 76 Appendices 79 Appendix A Numerical simulation method 80 A.1 Runge-Kutta method 80 A.2 Kahan summation 82 A.3 Simulation of the Kuramoto equation 82 Appendix B Asymmetric interaction-frequency correlated model 84 Appendix C Effective potential approaches for finite size systems 87 C.1 Random sampling of frequencies 87 C.2 Regular sampling of frequencies 91 C.3 Trapped at metastable states 92 Bibliography 100 Abstract in Korean 108Docto

Bee Hive Monitoring System - Solutions for the automated health monitoring

Cerca de um terço da produção global de alimentos depende da polinização das abelhas, tornando-as vitais para a economia mundial. No entanto, existem diversas ameaças à sobrevivência das espécies de abelhas, tais como incêndios florestais, stress humano induzido, subnutrição, poluição, perda de biodiversidade, agricultura intensiva e predadores como as vespas asiáticas. Destes problemas, pode-se observar um aumento da necessidade de soluções automatizadas que possam auxiliar na monitorização remota de colmeias de abelhas. O objetivo desta tese é desenvolver soluções baseadas em Aprendizagem Automática para problemas que podem ser identificados na apicultura, usando técnicas e conceitos de Deep Learning, Visão Computacional e Processamento de Sinal. Este documento descreve o trabalho da tese de mestrado, motivado pelo problema acima exposto, incluindo a revisão de literatura, análise de valor, design, planeamento de testes e validação e o desenvolvimento e estudo computacional das soluções. Concretamente, o trabalho desta tese de mestrado consistiu no desenvolvimento de soluções para três problemas – classificação da saúde de abelhas a partir de imagens e a partir de áudio, e deteção de abelhas e vespas asiáticas. Os resultados obtidos para a classificação da saúde das abelhas a partir de imagens foram significativamente satisfatórios, excedendo os que foram obtidos pela metodologia definida no trabalho base utilizado para a tarefa, que foi encontrado durante a revisão da literatura. No caso da classificação da saúde das abelhas a partir de áudio e da deteção de abelhas e vespas asiáticas, os resultados obtidos foram modestos e demonstram potencial aplicabilidade das respetivas metodologias desenvolvidas nos problemas-alvo. Pretende-se que as partes interessadas desta tese consigam obter informações, metodologias e perceções adequadas sobre o desenvolvimento de soluções de IA que possam ser integradas num sistema de monitorização da saúde de abelhas, incluindo custos e desafios inerentes à implementação das soluções. O trabalho futuro desta dissertação de mestrado consiste em melhorar os resultados dos modelos de classificação da saúde das abelhas a partir de áudio e de deteção de objetos, incluindo a publicação de artigos para obter validação pela comunidade científica.Up to one third of the global food production depends on the pollination of honey bees, making them vital for the world economy. However, between forest fires, human-induced stress, poor nutrition, pollution, biodiversity loss, intensive agriculture, and predators such as Asian Hornets, there are plenty of threats to the honey bee species’ survival. From these problems, a rise of the need for automated solutions that can aid with remote monitoring of bee hives can be observed. The goal of this thesis is to develop Machine Learning based solutions to problems that can be identified in beekeeping and apiculture, using Deep Learning, Computer Vision and Signal Processing techniques and concepts. The current document describes master thesis’ work, motivated from the above problem statement, including the literature review, value analysis, design, testing and validation planning and the development and computational study of the solutions. Specifically, this master thesis’ work consisted in developing solutions to three problems – bee health classification through images and audio, and bee and Asian wasp detection. Results obtained for the bee health classification through images were significantly satisfactory, exceeding those reported by the baseline work found during literature review. On the case of bee health classification through audio and bee and Asian wasp detection, these obtained results were modest and showcase potential applicability of the respective developed methodologies in the target problems. It is expected that stakeholders of this thesis obtain adequate information, methodologies and insights into the development of AI solutions that can be integrated in a bee health monitoring system, including inherent costs and challenges that arise with the implementation of the solutions. Future work of this master thesis consists in improving the results of the bee health classification through audio and the object detection models, including publishing of papers to seek validation by the scientific community

Sports Data Mining Technology Used in Basketball Outcome Prediction

Driven by the increasing comprehensive data in sports datasets and data mining technique successfully used in different area, sports data mining technique emerges and enables us to find hidden knowledge to impact the sport industry. In many instances, predicting the outcomes of sporting events has always been a challenging and attractive work and is therefore drawing a wide concern to conduct research in this field. This project focuses on using machine learning algorithms to build a model for predicting the NBA game outcomes and the algorithms involve Simple Logistics Classifier, Artificial Neural Networks, SVM and Naïve Bayes. In order to complete a convincing result, data of 5 regular NBA seasons was collected for model training and data of 1 NBA regular season was used as scoring dataset. After processes of automated data collection and cloud techniques enabled data management, a data mart containing NBA statistics data is built. Then machine learning models mentioned above is trained and tested by consuming data in the data mart. After applying scoring dataset to evaluate the model accuracy, Simple Logistics Classifier finally yields the best result with an accuracy of 69.67%. The results obtained are compared to other methods from different source. It was found that results of this project are more persuasive since such a vast quantity of data was applied in this project. Meanwhile, it can be referenced for the future work

Law Informs Code: A Legal Informatics Approach to Aligning Artificial Intelligence with Humans

We are currently unable to specify human goals and societal values in a way that reliably directs AI behavior. Law-making and legal interpretation form a computational engine that converts opaque human values into legible directives. "Law Informs Code" is the research agenda embedding legal knowledge and reasoning in AI. Similar to how parties to a legal contract cannot foresee every potential contingency of their future relationship, and legislators cannot predict all the circumstances under which their proposed bills will be applied, we cannot ex ante specify rules that provably direct good AI behavior. Legal theory and practice have developed arrays of tools to address these specification problems. For instance, legal standards allow humans to develop shared understandings and adapt them to novel situations. In contrast to more prosaic uses of the law (e.g., as a deterrent of bad behavior through the threat of sanction), leveraged as an expression of how humans communicate their goals, and what society values, Law Informs Code. We describe how data generated by legal processes (methods of law-making, statutory interpretation, contract drafting, applications of legal standards, legal reasoning, etc.) can facilitate the robust specification of inherently vague human goals. This increases human-AI alignment and the local usefulness of AI. Toward society-AI alignment, we present a framework for understanding law as the applied philosophy of multi-agent alignment. Although law is partly a reflection of historically contingent political power - and thus not a perfect aggregation of citizen preferences - if properly parsed, its distillation offers the most legitimate computational comprehension of societal values available. If law eventually informs powerful AI, engaging in the deliberative political process to improve law takes on even more meaning.Comment: Forthcoming in Northwestern Journal of Technology and Intellectual Property, Volume 2

The Baby project: processing character patterns in textual representations of language.

This thesis describes an investigation into a proposed theory of AI. The theory postulates that a machine can be programmed to predict aspects of human behaviour by selecting and processing stored, concrete examples of previously experienced patterns of behaviour. Validity is tested in the domain of natural language. Externalisations that model the resulting theory of NLP entail fuzzy components. Fuzzy formalisms may exhibit inaccuracy and/or over productivity. A research strategy is developed, designed to investigate this aspect of the theory. The strategy includes two experimental hypotheses designed to test, 1) whether the model can process simple language interaction, and 2) the effect of fuzzy processes on such language interaction. Experimental design requires three implementations, each with progressive degrees of fuzziness in their processes. They are respectively named: Nonfuzz Babe, CorrBab and FuzzBabe. Nonfuzz Babe is used to test the first hypothesis and all three implementations are used to test the second hypothesis. A system description is presented for Nonfuzz Babe. Testing the first hypothesis provides results that show NonfuzzBabe is able to process simple language interaction. A system description for CorrBabe and FuzzBabe is presented. Testing the second hypothesis, provides results that show a positive correlation between degree of fuzzy processes and improved simple language performance. FuzzBabe's ability to process more complex language interaction is then investigated and model-intrinsic limitations are found. Research to overcome this problem is designed to illustrate the potential of externalisation of the theory and is conducted less rigorously than previous part of this investigation. Augmenting FuzzBabe to include fuzzy evaluation of non-pattern elements of interaction is hypothesised as a possible solution. The term FuzzyBaby was coined for augmented implementation. Results of a pilot study designed to measure FuzzyBaby's reading comprehension are given. Little research has been conducted that investigates NLP by the fuzzy processing of concrete patterns in language. Consequently, it is proposed that this research contributes to the intellectual disciplines of NLP and AI in general

Comprehensive Taxonomies of Nature- and Bio-inspired Optimization: Inspiration versus Algorithmic Behavior, Critical Analysis and Recommendations

In recent years, a great variety of nature- and bio-inspired algorithms has been reported in the literature. This algorithmic family simulates different biological processes observed in Nature in order to efficiently address complex optimization problems. In the last years the number of bio-inspired optimization approaches in literature has grown considerably, reaching unprecedented levels that dark the future prospects of this field of research. This paper addresses this problem by proposing two comprehensive, principle-based taxonomies that allow researchers to organize existing and future algorithmic developments into well-defined categories, considering two different criteria: the source of inspiration and the behavior of each algorithm. Using these taxonomies we review more than three hundred publications dealing with nature-inspired and bio-inspired algorithms, and proposals falling within each of these categories are examined, leading to a critical summary of design trends and similarities between them, and the identification of the most similar classical algorithm for each reviewed paper. From our analysis we conclude that a poor relationship is often found between the natural inspiration of an algorithm and its behavior. Furthermore, similarities in terms of behavior between different algorithms are greater than what is claimed in their public disclosure: specifically, we show that more than one-third of the reviewed bio-inspired solvers are versions of classical algorithms. Grounded on the conclusions of our critical analysis, we give several recommendations and points of improvement for better methodological practices in this active and growing research field.Comment: 76 pages, 6 figure