Distribution-Independent Evolvability of Linear Threshold Functions

Valiant's (2007) model of evolvability models the evolutionary process of acquiring useful functionality as a restricted form of learning from random examples. Linear threshold functions and their various subclasses, such as conjunctions and decision lists, play a fundamental role in learning theory and hence their evolvability has been the primary focus of research on Valiant's framework (2007). One of the main open problems regarding the model is whether conjunctions are evolvable distribution-independently (Feldman and Valiant, 2008). We show that the answer is negative. Our proof is based on a new combinatorial parameter of a concept class that lower-bounds the complexity of learning from correlations. We contrast the lower bound with a proof that linear threshold functions having a non-negligible margin on the data points are evolvable distribution-independently via a simple mutation algorithm. Our algorithm relies on a non-linear loss function being used to select the hypotheses instead of 0-1 loss in Valiant's (2007) original definition. The proof of evolvability requires that the loss function satisfies several mild conditions that are, for example, satisfied by the quadratic loss function studied in several other works (Michael, 2007; Feldman, 2009; Valiant, 2010). An important property of our evolution algorithm is monotonicity, that is the algorithm guarantees evolvability without any decreases in performance. Previously, monotone evolvability was only shown for conjunctions with quadratic loss (Feldman, 2009) or when the distribution on the domain is severely restricted (Michael, 2007; Feldman, 2009; Kanade et al., 2010

Optimization Aspects of Carcinogenesis

Any process in which competing solutions replicate with errors and numbers of their copies depend on their respective fitnesses is the evolutionary optimization process. As during carcinogenesis mutated genomes replicate according to their respective qualities, carcinogenesis obviously qualifies as the evolutionary optimization process and conforms to common mathematical basis. The optimization view accents statistical nature of carcinogenesis proposing that during it the crucial role is actually played by the allocation of trials. Optimal allocation of trials requires reliable schemas' fitnesses estimations which necessitate appropriate, fitness landscape dependent, statistics of population. In the spirit of the applied conceptual framework, features which are known to decrease efficiency of any evolutionary optimization procedure (or inhibit it completely) are anticipated as "therapies" and reviewed. Strict adherence to the evolutionary optimization framework leads us to some counterintuitive implications which are, however, in agreement with recent experimental findings, such as sometimes observed more aggressive and malignant growth of therapy surviving cancer cells

The Emergence of Canalization and Evolvability in an Open-Ended, Interactive Evolutionary System

Natural evolution has produced a tremendous diversity of functional organisms. Many believe an essential component of this process was the evolution of evolvability, whereby evolution speeds up its ability to innovate by generating a more adaptive pool of offspring. One hypothesized mechanism for evolvability is developmental canalization, wherein certain dimensions of variation become more likely to be traversed and others are prevented from being explored (e.g. offspring tend to have similarly sized legs, and mutations affect the length of both legs, not each leg individually). While ubiquitous in nature, canalization almost never evolves in computational simulations of evolution. Not only does that deprive us of in silico models in which to study the evolution of evolvability, but it also raises the question of which conditions give rise to this form of evolvability. Answering this question would shed light on why such evolvability emerged naturally and could accelerate engineering efforts to harness evolution to solve important engineering challenges. In this paper we reveal a unique system in which canalization did emerge in computational evolution. We document that genomes entrench certain dimensions of variation that were frequently explored during their evolutionary history. The genetic representation of these organisms also evolved to be highly modular and hierarchical, and we show that these organizational properties correlate with increased fitness. Interestingly, the type of computational evolutionary experiment that produced this evolvability was very different from traditional digital evolution in that there was no objective, suggesting that open-ended, divergent evolutionary processes may be necessary for the evolution of evolvability.Comment: SI can be found at: http://www.evolvingai.org/files/SI_0.zi

Evolvability of Chaperonin Substrate Proteins

Molecular chaperones ensure that their substrate proteins reach the functional native state, and prevent their aggregation. Recently, an additional function was proposed for molecular chaperones: they serve as buffers (_capacitors_) for evolution by permitting their substrate proteins to mutate and at the same time still allowing them to fold productively.

Using pairwise alignments of _E. coli_ genes with genes from other gamma-proteobacteria, we showed that the described buffering effect cannot be observed among substrate proteins of GroEL, an essential chaperone in _E. coli_. Instead, we find that GroEL substrate proteins evolve less than other soluble _E. coli_ proteins. We analyzed several specific structural and biophysical properties of proteins to assess their influence on protein evolution and to find out why specifically GroEL substrates do not show the expected higher divergence from their orthologs.

Our results culminate in four main findings: *1.* We find little evidence that GroEL in _E. coli_ acts as a capacitor for evolution _in vivo_. *2.* GroEL substrates evolved less than other _E. coli_ proteins. *3.* Predominantly structural features appear to be a strong determinant of evolutionary rate. *4.* Besides size, hydrophobicity is a criterion for exclusion for a protein as a chaperonin substrate

Robust Learning under Strong Noise via {SQs}

This work provides several new insights on the robustness of Kearns' statistical query framework against challenging label-noise models. First, we build on a recent result by \cite{DBLP:journals/corr/abs-2006-04787} that showed noise tolerance of distribution-independently evolvable concept classes under Massart noise. Specifically, we extend their characterization to more general noise models, including the Tsybakov model which considerably generalizes the Massart condition by allowing the flipping probability to be arbitrarily close to $\frac{1}{2}$ for a subset of the domain. As a corollary, we employ an evolutionary algorithm by \cite{DBLP:conf/colt/KanadeVV10} to obtain the first polynomial time algorithm with arbitrarily small excess error for learning linear threshold functions over any spherically symmetric distribution in the presence of spherically symmetric Tsybakov noise. Moreover, we posit access to a stronger oracle, in which for every labeled example we additionally obtain its flipping probability. In this model, we show that every SQ learnable class admits an efficient learning algorithm with OPT + $\epsilon$ misclassification error for a broad class of noise models. This setting substantially generalizes the widely-studied problem of classification under RCN with known noise rate, and corresponds to a non-convex optimization problem even when the noise function -- i.e. the flipping probabilities of all points -- is known in advance

Designing Reusable and Run-Time Evolvable Scheduling Software

Scheduling processes have been applied to a large category of application areas such as processor scheduling in operating systems, assembly line balancing in factories, vehicle routing and scheduling in logistics and timetabling in public transportation, etc. In general, scheduling problems are not trivial to solve due to complex constraints. In this paper, we consider reusability and run-time evolvability as two important quality attributes to develop cost-effective software systems with schedulers. Although many proposals have been presented to enhance these quality attributes in general-purpose software development practices, there has been hardly any publication within the context of designing scheduling systems. This paper presents an application framework called First Scheduling Framework (FSF) to design and implement schedulers with a high-degree of reusability and run-time evolvability. The utility of the framework is demonstrated with a set of canonical examples and evolution scenarios. The framework is fully implemented and tested

Detection of new intentions from users for software service evolution in human-centric context-aware environments using Conditional Random Fields

The capability to accurately and efficiently obtain users’ new requirements is critical for software evolution, so that timely improvements can be made to systems to adapt to the rapidly changing environment. However, current software evolution cycles are often undesirably long because the elicitation of new requirements is mostly based on system performance or delayed user feedback and slow-paced manual analysis of requirements engineers. In this thesis, I propose a general methodology that employs Conditional Random Fields (CRF) as the mathematical foundation to provide quantitative exploration of users’ new intentions that often indicate their new requirements. My methodology is supposed to be applicable in context-aware software environments, and beneficial for discovering new requirements sooner and considerably shortening software evolution cycles. First of all, a situation-centric specification language – SiSL, is proposed to formalize the concepts and ontology of the application domains of our methodology. In SiSL, the domain of discourse is divided into five sorts of entities: action, desire, object, situation and situation-sequence. Another two important concepts, context and intention, are defined based on the five basic entities. A set of axioms are proposed to explain the relations among action, context values and desires. Based on the concepts and axioms in SiSL, a domain knowledge base which can completely describe and specify user’s behaviors and desires in human-centric context-aware environments can be constructed. To infer a user’s desire based on a peculiar form of observations and a specific detection mechanism for user’s new intentions, which may imply new requirements, the Conditional Random Fields (CRF) method is applied as a mathematical foundation to support my research work. In this thesis, the main part of a CRF model, a set of feature functions, specify the relations between observations (actions and context values) and human internal mental states (desires). To infer user’s desires, the CRF model accepts a sequence of observations as the input and calculates the score for each possible sequence-labeling, and outputs the sequence-labeling with the highest score as the inferred desire sequence. By using the CRF method, more accurate desire inference, the precondition for new intention detection, can be achieved compared with other statistical methods. To detect users’ potential new intentions, a CRF model which encodes users’ standard behavior patterns should be built as the metrics for outlier detection. The training data for building the standard CRF model are collected from observing user behaviors that are expected to conform to the system design. In the result of desire inference using the CRF model, the divergent behaviors will be labeled with desires in low confidence, and they can be singled out and analyzed for eliciting user’s potentially new intentions. Besides the divergent behaviors, user’s desire transitions and erroneous behaviors will also be analyzed for detecting new requirements or system drawbacks. The detected potential user’s new intention will be verified, analyzed and summarized to generate a formally new intention, which will drive system evolution through modifications or acquiring new functionalities to satisfy the new requirements. An experiment on a research library system has been conducted to demonstrate how to apply our methodology in detection of users’ new intentions and driving system evolution. Finally, this thesis discusses the threats to validity for our methodology and experiment