Plum: Prompt Learning using Metaheuristic

Since the emergence of large language models, prompt learning has become a popular method for optimizing and customizing these models. Special prompts, such as Chain-of-Thought, have even revealed previously unknown reasoning capabilities within these models. However, the progress of discovering effective prompts has been slow, driving a desire for general prompt optimization methods. Unfortunately, few existing prompt learning methods satisfy the criteria of being truly "general", i.e., automatic, discrete, black-box, gradient-free, and interpretable all at once. In this paper, we introduce metaheuristics, a branch of discrete non-convex optimization methods with over 100 options, as a promising approach to prompt learning. Within our paradigm, we test six typical methods: hill climbing, simulated annealing, genetic algorithms with/without crossover, tabu search, and harmony search, demonstrating their effectiveness in black-box prompt learning and Chain-of-Thought prompt tuning. Furthermore, we show that these methods can be used to discover more human-understandable prompts that were previously unknown, opening the door to a cornucopia of possibilities in prompt optimization. We release all the codes in \url{https://github.com/research4pan/Plum}

Investigating evolutionary computation with smart mutation for three types of Economic Load Dispatch optimisation problem

The Economic Load Dispatch (ELD) problem is an optimisation task concerned with how electricity generating stations can meet their customers’ demands while minimising under/over-generation, and minimising the operational costs of running the generating units. In the conventional or Static Economic Load Dispatch (SELD), an optimal solution is sought in terms of how much power to produce from each of the individual generating units at the power station, while meeting (predicted) customers’ load demands. With the inclusion of a more realistic dynamic view of demand over time and associated constraints, the Dynamic Economic Load Dispatch (DELD) problem is an extension of the SELD, and aims at determining the optimal power generation schedule on a regular basis, revising the power system configuration (subject to constraints) at intervals during the day as demand patterns change. Both the SELD and DELD have been investigated in the recent literature with modern heuristic optimisation approaches providing excellent results in comparison with classical techniques. However, these problems are defined under the assumption of a regulated electricity market, where utilities tend to share their generating resources so as to minimise the total cost of supplying the demanded load. Currently, the electricity distribution scene is progressing towards a restructured, liberalised and competitive market. In this market the utility companies are privatised, and naturally compete with each other to increase their profits, while they also engage in bidding transactions with their customers. This formulation is referred to as: Bid-Based Dynamic Economic Load Dispatch (BBDELD). This thesis proposes a Smart Evolutionary Algorithm (SEA), which combines a standard evolutionary algorithm with a “smart mutation” approach. The so-called ‘smart’ mutation operator focuses mutation on genes contributing most to costs and penalty violations, while obeying operational constraints. We develop specialised versions of SEA for each of the SELD, DELD and BBDELD problems, and show that this approach is superior to previously published approaches in each case. The thesis also applies the approach to a new case study relevant to Nigerian electricity deregulation. Results on this case study indicate that our SEA is able to deal with larger scale energy optimisation tasks

Genetic learning as an explanation of stylized facts of foreign exchange markets

This paper revisits the Kareken-Wallace model of exchange rate formation in a two-country overlapping generations world. Following the seminal paper by Arifovic (Journal of Political Economy, 104, 1996, 510 – 541) we investigate a dynamic version of the model in which agents? decision rules are updated using genetic algorithms. Our main interest is in whether the equilibrium dynamics resulting from this learning process helps to explain the main stylized facts of free-floating exchange rates (unit roots in levels together with fat tails in returns and volatility clustering). Our time series analysis of simulated data indicates that for particular parameterizations, the characteristics of the exchange rate dynamics are, in fact, very similar to those of empirical data. The similarity appears to be quite insensitive with respect to some of the ingredients of the GA algorithm (i.e. utility-based versus rank-based or tournament selection, binary or real coding). However, appearance or not of realistic time series characteristics depends crucially on the mutation probability (which should be low) and the number of agents (not more than about 1000). With a larger population, this collective learning dynamics looses its realistic appearance and instead exhibits regular periodic oscillations of the agents? choice variables. --learning , genetic algorithms , exchange rate dynamics

Advances in Evolutionary Algorithms

With the recent trends towards massive data sets and significant computational power, combined with evolutionary algorithmic advances evolutionary computation is becoming much more relevant to practice. Aim of the book is to present recent improvements, innovative ideas and concepts in a part of a huge EA field

Genetic learning as an explanation of stylized facts of foreign exchange markets

This paper revisits the Kareken-Wallace model of exchange rate formation in a two-country overlapping generations world. Following the seminal paper by Arifovic (Journal of Political Economy, 104, 1996, 510-541) we investigate a dynamic version of the model in which agents' decision rules are updated using genetic algorithms. Our main interest is in whether the equilibrium dynamics resulting from this learning process helps to explain the main stylized facts of free-floating exchange rates (unit roots in levels together with fat tails in returns and volatility clustering). Our time series analysis of simulated data indicates that for particular parameterizations, the characteristics of the exchange rate dynamics are, in fact, very similar to those of empirical data. The similarity appears to be quite insensitive with respect to some of the ingredients of the GA algorithm (i.e. utilitybased versus rank-based or tournament selection, binary or real coding). However, appearance or not of realistic time series characteristics depends crucially on the mutation probability (which should be low) and the number of agents (not more than about 1000). With a larger population, this collective learning dynamics looses its realistic appearance and instead exhibits regular periodic oscillations of the agents' choice variables. -- Dieses Papier betrachtet das Kareken-Wallace-Modell für die Wechselkursbildung in einer Welt mit 2 Ländern und sich überlappenden Generationen. In der Nachfolge des zukunftsweisenden Papiers von Arifovic (1996) untersuchen wir eine dynamische Version des Modells bei dem die Entscheidungsregeln mithilfe genetischer Algorithmen jeweils aktualisiert werden. Unser Hauptinteresse geht dahin, herauszufinden, ob die Gleichgewichtsdynamik, die aus diesem Lernprozess resultiert, dabei helfen kann, die wichtigsten stilisierten Fakten von flexiblen Wechselkursen zu erklären (Einheitswurzeln bei den Niveaus mit dicken Enden der Ertragsverteilung und Klumpenbildung bei den Volatilitäten). Unsere Analyse simulierter Daten weist darauf hin, dass für bestimmte Parametrisierungen der Charakter der Wechselkursdynamik tatsächlich dem von empirischen Daten sehr ähnlich ist. Die Ähnlichkeit scheint sehr wenig von speziellen Eigenschaften des gewählten GA-Algorithmus abzuhängen (z. B. nutzenbasiert versus rangbasiert, binäre oder reale Kodierung). Dagegen ist die Mutationswahrscheinlichkeit (die niedrig sein sollte) und die Anzahl der Agenten (die nicht größer als 1000 sein sollte) wichtig. Mit mehr Teilnehmern verliert die kollektive Lerndynamik ihr realistisches Aussehen und es kommt zu regelmäßigen periodischen Schwankungen bei den Variablen, die die Agenten auswählen.Learning,Genetic algorithms,Exchange rate dynamics

A hybrid genetic algorithm and tabu search approach for post enrolment course timetabling

Copyright @ Springer Science + Business Media. All rights reserved.The post enrolment course timetabling problem (PECTP) is one type of university course timetabling problems, in which a set of events has to be scheduled in time slots and located in suitable rooms according to the student enrolment data. The PECTP is an NP-hard combinatorial optimisation problem and hence is very difficult to solve to optimality. This paper proposes a hybrid approach to solve the PECTP in two phases. In the first phase, a guided search genetic algorithm is applied to solve the PECTP. This guided search genetic algorithm, integrates a guided search strategy and some local search techniques, where the guided search strategy uses a data structure that stores useful information extracted from previous good individuals to guide the generation of offspring into the population and the local search techniques are used to improve the quality of individuals. In the second phase, a tabu search heuristic is further used on the best solution obtained by the first phase to improve the optimality of the solution if possible. The proposed hybrid approach is tested on a set of benchmark PECTPs taken from the international timetabling competition in comparison with a set of state-of-the-art methods from the literature. The experimental results show that the proposed hybrid approach is able to produce promising results for the test PECTPs.This work was supported by the Engineering and Physical Sciences Research Council (EPSRC) of UK under Grant EP/E060722/01 and Grant EP/E060722/02

New Swarm-Based Metaheuristics for Resource Allocation and Schwduling Problems

Tesis doctoral inédita leída en la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Ingeniería Informática. Fecha de lectura : 10-07-2017Esta tesis tiene embargado el acceso al texto completo hasta el 10-01-201

Applications of Artificial Intelligence in Power Systems

Artificial intelligence tools, which are fast, robust and adaptive can overcome the drawbacks of traditional solutions for several power systems problems. In this work, applications of AI techniques have been studied for solving two important problems in power systems. The first problem is static security evaluation (SSE). The objective of SSE is to identify the contingencies in planning and operations of power systems. Numerical conventional solutions are time-consuming, computationally expensive, and are not suitable for online applications. SSE may be considered as a binary-classification, multi-classification or regression problem. In this work, multi-support vector machine is combined with several evolutionary computation algorithms, including particle swarm optimization (PSO), differential evolution, Ant colony optimization for the continuous domain, and harmony search techniques to solve the SSE. Moreover, support vector regression is combined with modified PSO with a proposed modification on the inertia weight in order to solve the SSE. Also, the correct accuracy of classification, the speed of training, and the final cost of using power equipment heavily depend on the selected input features. In this dissertation, multi-object PSO has been used to solve this problem. Furthermore, a multi-classifier voting scheme is proposed to get the final test output. The classifiers participating in the voting scheme include multi-SVM with different types of kernels and random forests with an adaptive number of trees. In short, the development and performance of different machine learning tools combined with evolutionary computation techniques have been studied to solve the online SSE. The performance of the proposed techniques is tested on several benchmark systems, namely the IEEE 9-bus, 14-bus, 39-bus, 57-bus, 118-bus, and 300-bus power systems. The second problem is the non-convex, nonlinear, and non-differentiable economic dispatch (ED) problem. The purpose of solving the ED is to improve the cost-effectiveness of power generation. To solve ED with multi-fuel options, prohibited operating zones, valve point effect, and transmission line losses, genetic algorithm (GA) variant-based methods, such as breeder GA, fast navigating GA, twin removal GA, kite GA, and United GA are used. The IEEE systems with 6-units, 10-units, and 15-units are used to study the efficiency of the algorithms

Applications of Artificial Intelligence in Power Systems

Artificial intelligence tools, which are fast, robust and adaptive can overcome the drawbacks of traditional solutions for several power systems problems. In this work, applications of AI techniques have been studied for solving two important problems in power systems. The first problem is static security evaluation (SSE). The objective of SSE is to identify the contingencies in planning and operations of power systems. Numerical conventional solutions are time-consuming, computationally expensive, and are not suitable for online applications. SSE may be considered as a binary-classification, multi-classification or regression problem. In this work, multi-support vector machine is combined with several evolutionary computation algorithms, including particle swarm optimization (PSO), differential evolution, Ant colony optimization for the continuous domain, and harmony search techniques to solve the SSE. Moreover, support vector regression is combined with modified PSO with a proposed modification on the inertia weight in order to solve the SSE. Also, the correct accuracy of classification, the speed of training, and the final cost of using power equipment heavily depend on the selected input features. In this dissertation, multi-object PSO has been used to solve this problem. Furthermore, a multi-classifier voting scheme is proposed to get the final test output. The classifiers participating in the voting scheme include multi-SVM with different types of kernels and random forests with an adaptive number of trees. In short, the development and performance of different machine learning tools combined with evolutionary computation techniques have been studied to solve the online SSE. The performance of the proposed techniques is tested on several benchmark systems, namely the IEEE 9-bus, 14-bus, 39-bus, 57-bus, 118-bus, and 300-bus power systems. The second problem is the non-convex, nonlinear, and non-differentiable economic dispatch (ED) problem. The purpose of solving the ED is to improve the cost-effectiveness of power generation. To solve ED with multi-fuel options, prohibited operating zones, valve point effect, and transmission line losses, genetic algorithm (GA) variant-based methods, such as breeder GA, fast navigating GA, twin removal GA, kite GA, and United GA are used. The IEEE systems with 6-units, 10-units, and 15-units are used to study the efficiency of the algorithms