Mutation and optimal search of sequences in nested Hamming spaces

A representation of evolutionary systems is defined by sequences in nested Hamming spaces, which is analogous to variable length coding. Asexual reproduction is considered as a process of finding optimal codes, and conditions are formulated under which the optimal search operator is a simple random mutation, corresponding to binomial or Poisson process. Transition probability between spheres around an optimal sequence in each Hamming space is derived and used for optimal control of mutation rate. Several control functions are discussed, including a minimal information control that has a number of interesting properties. The theory makes a number of predictions about variability of length and mutation of DNA sequences in biological organisms

Bounds of optimal learning.

Learning is considered as a dynamic process described by a trajectory on a statistical manifold, and a topology is introduced defining trajectories continuous in information. The analysis generalises the application of Orlicz spaces in non-parametric information geometry to topological function spaces with asymmetric gauge functions (e.g. quasi-metric spaces defined in terms of KL divergence). Optimality conditions are formulated for dynamical constraints, and two main results are outlined: 1) Parametrisation of optimal learning trajectories from empirical constraints using generalised characteristic potentials; 2) A gradient theorem for the potentials defining optimal utility and information bounds of a learning system. These results not only generalise some known relations of statistical mechanics and variational methods in information theory, but also can be used for optimisation of the exploration-exploitation balance in online learning systems

The duality of utility and information in optimally learning systems.

The paper considers learning systems as optimisation systems with dynamical information constraints, and general optimality conditions are derived using the duality between the space of utility functions and probability measures. The increasing dynamics of the constraints is used to parametrise the optimal solutions which form a trajectory in the space of probability measures. Stochastic processes following such trajectories describe systems achieving the maximum possible utility gain with respect to a given information. The theory is discussed on examples for finite and uncountable sets and in relation to existing applications and cognitive models of learning

Relation between the Kantorovich-Wasserstein metric and the Kullback-Leibler divergence

We discuss a relation between the Kantorovich-Wasserstein (KW) metric and the Kullback-Leibler (KL) divergence. The former is defined using the optimal transport problem (OTP) in the Kantorovich formulation. The latter is used to define entropy and mutual information, which appear in variational problems to find optimal channel (OCP) from the rate distortion and the value of information theories. We show that OTP is equivalent to OCP with one additional constraint fixing the output measure, and therefore OCP with constraints on the KL-divergence gives a lower bound on the KW-metric. The dual formulation of OTP allows us to explore the relation between the KL-divergence and the KW-metric using decomposition of the former based on the law of cosines. This way we show the link between two divergences using the variational and geometric principles

Towards a theory of decision-making with paradoxes.

Human subjects often violate the rational decision-making theory, which is based on the notion of expected utility and axioms of choice (Neuman & Morgenstern, 1944; Savage, 1954). The counterexamples, suggested by Allais (1953) and Ellsberg (1961), deserve special attention because they point at our lack of understanding of how humans make decisions. The paradoxes of decision-making are particularly important for the ACT-R theory which currently relies on expected utility. The paper presents two alternative methods: a random prediction method that uses subsymbolic computations and a method that uses symbolic reasoning for qualitative decision-making. Both methods are tested on ACT-R models of the paradoxes, and the advantages of each method are discussed

Acting irrationally to improve performance in stochastic worlds

Despite many theories and alogorithms for decision-making, after estimating the utility function the choice is usually made by maximising its expected value (the max EU principle). This traditional and 'rational' conclusion of the decision-making process is compared in this paper with several 'irrational' techniques that make choice in Monte-Carlo fashion. The comparison is made by evaluating the performance of simple decision-theoretic agents in stochastic environments. It is shown that not only the random choice strategies can achieve performance comparable to the max EU method, but under certain conditions the Monte-Carlo choice methods perform almost two times better than the max EU. The paper concludes by quoting evidence from recent cognitive modelling works as well as the famous decision-making paradoxes

On relation between emotion and entropy

The ways of modelling some of the most profound effects of emotion and arousal on cognition are discussed. Entropy reduction is used to measure quantitatively the learning spedd in in a cognitive model under different parameters' conditions. It is noticed that some settings facilitate the learning in particular stages of problem solving more than others. The entropy feedback is used to control these parameters and strategy, which in turn improves greatly the learning in the model as well as the model match with the data. This result may explain the reasons behind some of the neurobiological changes, associated with emotion and its control of the decision making stragy and behaviour

Optimal measures and Markov transition kernels

We study optimal solutions to an abstract optimization problem for measures, which is a generalization of classical variational problems in information theory and statistical physics. In the classical problems, information and relative entropy are defined using the Kullback-Leibler divergence, and for this reason optimal measures belong to a one-parameter exponential family. Measures within such a family have the property of mutual absolute continuity. Here we show that this property characterizes other families of optimal positive measures if a functional representing information has a strictly convex dual. Mutual absolute continuity of optimal probability measures allows us to strictly separate deterministic and non-deterministic Markov transition kernels, which play an important role in theories of decisions, estimation, control, communication and computation. We show that deterministic transitions are strictly sub-optimal, unless information resource with a strictly convex dual is unconstrained. For illustration, we construct an example where, unlike non-deterministic, any deterministic kernel either has negatively infinite expected utility (unbounded expected error) or communicates infinite information

Conflict resolution by random estimated costs

Conflict resolution is an important part of many intelligent systems such as production systems, planning tools and cognitive architectures. For example, the ACT-R cognitive architecture (Anderson and Lebiere, 1998) uses a powerful conflict resolution theory that allowed for modelling many characteristics of human decision making. The results of more recent works, however, pointed to the need of revisiting the conflict resolution theory of ACT-R to incorporate more dynamics. In the proposed theory the conflict is resolved using the estimates of the expected costs of production rules. The method has been implemented as a stand alone search program as well as an add-on to the ACT-R architecture replacing the standard mechanism. The method expresses more dynamic and adaptive behaviour. The performance of the algorithm shows that it can be successfully used as a search and optimisation technique

Nondemolition Principle of Quantum Measurement Theory

We give an explicit axiomatic formulation of the quantum measurement theory which is free of the projection postulate. It is based on the generalized nondemolition principle applicable also to the unsharp, continuous-spectrum and continuous-in-time observations. The "collapsed state-vector" after the "objectification" is simply treated as a random vector of the a posteriori state given by the quantum filtering, i.e., the conditioning of the a priori induced state on the corresponding reduced algebra. The nonlinear phenomenological equation of "continuous spontaneous localization" has been derived from the Schroedinger equation as a case of the quantum filtering equation for the diffusive nondemolition measurement. The quantum theory of measurement and filtering suggests also another type of the stochastic equation for the dynamical theory of continuous reduction, corresponding to the counting nondemolition measurement, which is more relevant for the quantum experiments.Comment: 23 pages. See also related papers at http://www.maths.nott.ac.uk/personal/vpb/research/mes_fou.html and http://www.maths.nott.ac.uk/personal/vpb/research/cau_idy.htm