Evolving stochastic learning algorithm based on Tsallis entropic index

In this paper, inspired from our previous algorithm, which was based on the theory of Tsallis statistical mechanics, we develop a new evolving stochastic learning algorithm for neural networks. The new algorithm combines deterministic and stochastic search steps by employing a different adaptive stepsize for each network weight, and applies a form of noise that is characterized by the nonextensive entropic index q, regulated by a weight decay term. The behavior of the learning algorithm can be made more stochastic or deterministic depending on the trade off between the temperature T and the q values. This is achieved by introducing a formula that defines a time-dependent relationship between these two important learning parameters. Our experimental study verifies that there are indeed improvements in the convergence speed of this new evolving stochastic learning algorithm, which makes learning faster than using the original Hybrid Learning Scheme (HLS). In addition, experiments are conducted to explore the influence of the entropic index q and temperature T on the convergence speed and stability of the proposed method

A characterization of the scientific impact of Brazilian institutions

In this paper we studied the research activity of Brazilian Institutions for all sciences and also their performance in the area of physics between 1945 and December 2008. All the data come from the Web of Science database for this period. The analysis of the experimental data shows that, within a nonextensive thermostatistical formalism, the Tsallis \emph{q}-exponential distribution $N(c)$ can constitute a new characterization of the research impact for Brazilian Institutions. The data examined in the present survey can be fitted successfully by applying a universal curve namely, $N(c) \propto 1/[1+(q-1) c/T]^{\frac{1}{q-1}}$ with $q\simeq 4/3$ for {\it all} the available citations $c$, $T$ being an "effective temperature". The present analysis ultimately suggests that via the "effective temperature" $T$, we can provide a new performance metric for the impact level of the research activity in Brazil, taking into account the number of the publications and their citations. This new performance metric takes into account the "quantity" (number of publications) and the "quality" (number of citations) for different Brazilian Institutions. In addition we analyzed the research performance of Brazil to show how the scientific research activity changes with time, for instance between 1945 to 1985, then during the period 1986-1990, 1991-1995, and so on until the present. Finally, this work intends to show a new methodology that can be used to analyze and compare institutions within a given country.Comment: 7 pages, 5 figure

Nonextensive statistics: Theoretical, experimental and computational evidences and connections

The domain of validity of standard thermodynamics and Boltzmann-Gibbs statistical mechanics is discussed and then formally enlarged in order to hopefully cover a variety of anomalous systems. The generalization concerns {\it nonextensive} systems, where nonextensivity is understood in the thermodynamical sense. This generalization was first proposed in 1988 inspired by the probabilistic description of multifractal geometries, and has been intensively studied during this decade. In the present effort, after introducing some historical background, we briefly describe the formalism, and then exhibit the present status in what concerns theoretical, experimental and computational evidences and connections, as well as some perspectives for the future. In addition to these, here and there we point out various (possibly) relevant questions, whose answer would certainly clarify our current understanding of the foundations of statistical mechanics and its thermodynamical implicationsComment: 15 figure

QESK: Quantum-based Entropic Subtree Kernels for Graph Classification

In this paper, we propose a novel graph kernel, namely the Quantum-based Entropic Subtree Kernel (QESK), for Graph Classification. To this end, we commence by computing the Average Mixing Matrix (AMM) of the Continuous-time Quantum Walk (CTQW) evolved on each graph structure. Moreover, we show how this AMM matrix can be employed to compute a series of entropic subtree representations associated with the classical Weisfeiler-Lehman (WL) algorithm. For a pair of graphs, the QESK kernel is defined by computing the exponentiation of the negative Euclidean distance between their entropic subtree representations, theoretically resulting in a positive definite graph kernel. We show that the proposed QESK kernel not only encapsulates complicated intrinsic quantum-based structural characteristics of graph structures through the CTQW, but also theoretically addresses the shortcoming of ignoring the effects of unshared substructures arising in state-of-the-art R-convolution graph kernels. Moreover, unlike the classical R-convolution kernels, the proposed QESK can discriminate the distinctions of isomorphic subtrees in terms of the global graph structures, theoretically explaining the effectiveness. Experiments indicate that the proposed QESK kernel can significantly outperform state-of-the-art graph kernels and graph deep learning methods for graph classification problems

Self-adaptation of mutation distribution in evolution strategies for dynamic optimization problems

Copyright @ IOS Press. All Rights Reserved.Evolution strategies with q-Gaussian mutation, which allows the self-adaptation of the mutation distribution shape, is proposed for dynamic optimization problems in this paper. In the proposed method, a real parameter q, which allows to smoothly control the shape of the mutation distribution, is encoded in the chromosome of the individuals and is allowed to evolve. In the experimental study, the q-Gaussian mutation is compared to Gaussian and Cauchy mutation on experiments generated from the simulation of evolutionary robots and on dynamic optimization problems generated by the Moving Peaks generator

Appraisal of the self-organization and evolutionary dynamics of seismicity based on (non-extensive) statistical physics and complexity science methods

Θεμελιώδης πρόκληση σε πολλά επιστημονικά πεδία αποτελεί ο καθορισμός κανονικοτήτων και νόμων ανωτέρας κλίμακας σε σχέση με την υπάρχουσα γνώση για φαινόμενα κατωτέρας κλίμακας. Είναι πλέον αποδεκτό ότι o ενεργός τεκτονικός ιστός αποτελεί ένα κρίσιμο πολύπλοκο σύστημα, αν και δεν έχει ακόμη οριστικοποιηθεί αν είναι στατικό, δυναμικό/εξελικτικό, ή ένας χρονικά εξαρτημένος συνδυασμός αμφοτέρων. Σε κάθε περίπτωση, τα κρίσιμα συστήματα χαρακτηρίζονται από μορφοκλασματική ή πολυ-μορφοκλασματική κατανομή των στοιχείων τους, ισχυρές αλληλεπιδράσεις μεταξύ των κοντινών και μακρινών γειτόνων και διακοπτόμενη (ασυνεχή) έκφρασή τους. Οι ιδιότητες αυτές μπορούν να μελετηθούν με όρους Μη Εκτατικής Στατιστικής Φυσικής (ΜΕΣΦ). Πέραν του ρυθμού έκλυσης ενέργειας που εκφράζεται μέσω του μεγέθους (Μ), μέτρο των πιθανών συσχετίσεων αποτελεί ο παρέλθων χρόνος (Δt) και η υποκεντρική απόσταση (Δd) μεταξύ αλλεπάλληλων σεισμών πάνω από ένα κατώφλι μεγέθους σε μια περιοχή. Πρόσφατες έρευνες έδειξαν ότι, εάν οι κατανομές μεγέθους (Μ), χρονικής (Δt) και χωρικής (Δd) εξάρτησης μεταξύ διαδοχικών σεισμών θεωρηθούν ανεξάρτητες έτσι ώστε η από κοινού πιθανότητα p(M, Δt, Δd) να παραγοντοποιείται σε p(MUΔtUΔd) = p(M) p(Δt) p(Δd), τότε η συχνότητα εμφάνισης ενός σεισμού εξαρτάται πολλαπλώς όχι μόνο από το μέγεθος όπως πρόβλεπει ο νόμος Gutenberg – Richter αλλά και από τη χρονική και χωρική εξάρτηση διαδοχικών σεισμών. Αυτό, με τη σειρά του, σημαίνει ότι η αυτο-οργάνωση της σεισμικότητας θα πρέπει να εκδηλώνεται μέσω μιας συγκεκριμένης στατιστικής συμπεριφοράς της χρονικής και χωρικής εξάρτησης της (κατανομές νόμων δύναμης). Στην παρούσα διατριβή θα επιχειρηθεί η περιγραφή της σεισμικότητας με όρους ΜΕΣΦ, σε σεισμογενετικά συστήματα κατά μήκος του ορίου πλακών του ΒΑ-Β Ειρηνικού και της Βορείου Αμερικής, καθώς και στο σεισμογενετικό σύστημα του ελλαδικού χώρου-Δυτικής Τουρκίας. Η ανάλυση πραγματοποιείται σε πλήρης και ομαδοποιημένους καταλόγους σεισμών, όπου οι μετασεισμοί έχουν αφαιρεθεί με τη στοχαστική μέθοδο απομαδοποίησης του Zhuang et al., (2002). Η στατιστική συμπεριφορά της σεισμικότητας υποδεικνύει ότι η επιφανειακή σεισμικότητα των συστημάτων που μελετώνται είναι υποεκτατική, χαρακτηρίζεται από μακράς εμβέλειας συσχετίσεις και για το λόγο αυτό είναι αυτο-οργανωμένη και πιθανόν κρίσιμη. Ο βαθμός της υπο-εκτακτικότητας δεν είναι ομοιόμορφος, ούτε σταθερός, αλλά διαφέρει δυναμικά από σύστημα σε σύστημα, ενίοτε διαφέρει στη χρονική εξέλιξη και μπορεί να παρουσιάζει κυκλικότητα. Το μόνο σύστημα βαθειάς δομής (σεισμικότητα σε μεγάλα εστιακά βάθη) που εξετάζεται εδώ - η Αλεούτια ζώνη υποβύθισης- φαίνεται να παρουσιάζει στατιστική που περιγράφεται με όρους κατανομής Poisson (απουσία συσχέτισης). Τα αποτελέσματα που προκύπτουν υποδεικνύουν ότι η ΜΕΣΦ αποτελεί ένα εξαιρετικό εργαλείο για την φυσική περιγραφή της σεισμικότητας σε διάφορα σεισμογενετικά περιβάλλοντα. Ο μη εκτατικός φορμαλισμός θεωρείται το κατάλληλο μεθοδολογικό εργαλείο για να περιγράψει φυσικά συστήματα που δε βρίσκονται σε ισορροπία και έχουν μεγάλη μεταβλητότητα και πολυκλασματική δομή όπως η σεισμικότητα.A fundamental challenge in many scientific fields is to define norms and laws of higher-order in relation to the existing knowledge about phenomena of lower-order. It has been long suggested that the active tectonic grain comprises a self-organized complex system, therefore its expression (seismicity) should be manifested in the temporal and spatial statistics of energy release rates, and exhibit memory due to long-range interactions in a fractal-like space-time. Such attributes can be properly understood in terms of Non-Extensive Statistical Physics (NESP) In addition to energy release rates expressed by the magnitude M, measures of the temporal and spatial interactions are the time (Δt) and hypocentral distance (Δd) between consecutive events. Recent work indicated that if the distributions of M, Δt and Δd are independent so that the joint probability p(M, Δt, Δd) factorizes into the probabilities of M, Δt and Δd, i.e. p(MUΔtUΔd) = p(M) p(Δt) p(Δd), then the frequency of earthquake occurrence is multiply related, not only to magnitude as the celebrated Gutenberg – Richter law predicts, but also to interevent time and distance by means of well-defined power-laws consistent with NESP. The present work applies these concepts to investigate the dynamics of seismogenetic systems along the NE – N boundary of the Pacific and North American plates and the seismogenic zones of Greece – Western Turkey. The analysis is conducted to full and declustered (reduced) catalogues where the aftreshocks are removed by the stochasting declustering method of Zhuang et al., 2002.The statistical behaviour of seismicity suggests that crustal seismogenetic systems along the Pacific–North American plate boundaries in California, the seismogenic zones of Greece – Western Turkey, Alaska and the Aleutian Arc are invariably sub-extensive; they exhibit prominent operative long-range interaction and long-term memory, therefore they are self-organized and possibly critical. The degree of sub-extensivity is neither uniform, nor stationary but varies dynamically between systems and may also vary with time, or in cycles. The only sub-crustal system studied herein (Aleutian Subduction) appears to be Poissonian. The results are consistent with simulations of small-world fault networks in which free boundary conditions at the edges, (i.e. at the surface) allow for self-organization and criticality to develop, and fixed boundary conditions within, (i.e. at depth), do not. The results indicate that NESP is an excellent natural descriptor of earthquake statistics and appears to apply to the seismicity observed in different seismogenetic environments. The NESP formalism, although far from having answered questions and debates on the statistical physics of earthquakes, appears to be an effective and insightful tool in the investigation of seismicity and its associated complexity

Entropy-driven dynamics and robust learning procedures in games

In this paper, we introduce a new class of game dynamics made of a pay-off replicator-like term modulated by an entropy barrier which keeps players away from the boundary of the strategy space. We show that these {\it entropy-driven} dynamics are equivalent to players computing a score as their on-going exponentially discounted cumulative payoff and then using a quantal choice model on the scores to pick an action. This dual perspective on {\it entropy-driven} dynamics helps us to extend the folk theorem on convergence to quantal response equilibria to this case, for potential games. It also provides the main ingredients to design a discrete time effective learning algorithm that is fully distributed and only requires partial information to converge to QRE. This convergence is resilient to stochastic perturbations and observation errors and does not require any synchronization between the players