Architecting system of systems: artificial life analysis of financial market behavior

This research study focuses on developing a framework that can be utilized by system architects to understand the emergent behavior of system architectures. The objective is to design a framework that is modular and flexible in providing different ways of modeling sub-systems of System of Systems. At the same time, the framework should capture the adaptive behavior of the system since evolution is one of the key characteristics of System of Systems. Another objective is to design the framework so that humans can be incorporated into the analysis. The framework should help system architects understand the behavior as well as promoters or inhibitors of change in human systems. Computational intelligence tools have been successfully used in analysis of Complex Adaptive Systems. Since a System of Systems is a collection of Complex Adaptive Systems, a framework utilizing combination of these tools can be developed. Financial markets are selected to demonstrate the various architectures developed from the analysis framework --Introduction, page 3

A time series classifier

A time series is a sequence of data measured at successive time intervals. Time series analysis refers to all of the methods employed to understand such data, either with the purpose of explaining the underlying system producing the data or to try to predict future data points in the time series...An evolutionary algorithm is a non-deterministic method of searching a solution space, and modeled after biological evolutionary processes. A learning classifier system (LCS) is a form of evolutionary algorithm that operates on a population of mapping rules. We introduce the time series classifier TSC, a new type of LCS that allows for the modeling and prediction of time series data, derived from Wilson\u27s XCSR, an LCS designed for use with real-valued inputs. Our method works by modifying the makeup of the rules in the LCS so that they are suitable for use on a time series...We tested TSC on real-world historical stock data --Abstract, page iii

XCS Algorithms for a Linear Combination of Discounted and Undiscounted Reward Markovian Decision Processes

RÉSUMÉ : Plusieurs études ont montré que combiner certains prédicteurs ensemble peut améliorer la justesse de la prédiction dans certains domaines comme la psychologie, les statistiques ou les sciences du management. Toutefois, aucune de ces études n'ont testé la combinaison de techniques d'apprentissage par renforcement. Notre étude vise à développer un algorithme basé sur deux algorithmes qui sont des formes approximatives d'apprentissage par renforcement répétés dans XCS. Cet algorithme, MIXCS, est une combinaison des techniques de Q-learning et de R-learning pour calculer la combinaison linéaire du payoff résultant des actions de l'agent, et aussi la correspondance entre la prédiction au niveau du système et la valeur réelle des actions de l'agent. MIXCS fait une prévision du payoff espéré pour chacune des actions disponibles pour l'agent. Nous avons testé MIXCS dans deux environnements à deux dimensions, Environment1 et Environment2, qui reproduisent les actions possibles dans un marché financier (acheter, vendre, ne rien faire) pour évaluer les performances d'un agent qui veut obtenir un profit espéré. Nous avons calculé le payoff optimal moyen dans nos deux environnements et avons comparé avec les résultats obtenus par MIXCS. Nous avons obtenu deux résultats. En premier, les résultats de MIXCS sont semblables au payoff optimal moyen pour Environments1, mais pas pour Environment2. Deuxièmement, l'agent obtient le payoff optimal moyen quand il prend l'action "vendre" dans les deux environnements.----------ABSTRACT : Many studies have shown that combining individual predictors improved the accuracy of predictions in different domains such as psychology, statistics and management sciences. However, these studies have not tested the combination of reinforcement learning techniques. This study aims to develop an algorithm based on two iterative approximate forms of reinforcement learning algorithm in XCS. This algorithm, named MIXCS, is a combination of Q-learning and R-learning techniques to compute the linear combination payoff and the correspondence between the system prediction and the action value. As such, MIXCS predicts the payoff to be expected for each possible action. We test MIXCS in two two-dimensional grids called Environment1 and Environment2, which represent financial markets actions of buying, selling and holding to evaluate the performance of an agent as a trader to gain the desired profit. We calculate the optimum average payoff to predict the value of the next movement in both Environment1 and Environment2 and compare the results with those obtained with MIXCS. The results show that the performance of MIXCS is close to optimum average reward in Environment1, but not in Environment2. Also, the agent reaches the maximum reward by taking selling actions in both Environments

INVESTIGATIONS INTO THE COGNITIVE ABILITIES OF ALTERNATE LEARNING CLASSIFIER SYSTEM ARCHITECTURES

The Learning Classifier System (LCS) and its descendant, XCS, are promising paradigms for machine learning design and implementation. Whereas LCS allows classifier payoff predictions to guide system performance, XCS focuses on payoff-prediction accuracy instead, allowing it to evolve optimal classifier sets in particular applications requiring rational thought. This research examines LCS and XCS performance in artificial situations with broad social/commercial parallels, created using the non-Markov Iterated Prisoner\u27s Dilemma (IPD) game-playing scenario, where the setting is sometimes asymmetric and where irrationality sometimes pays. This research systematically perturbs a conventional IPD-playing LCS-based agent until it results in a full-fledged XCS-based agent, contrasting the simulated behavior of each LCS variant in terms of a number of performance measures. The intent is to examine the XCS paradigm to understand how it better copes with a given situation (if it does) than the LCS perturbations studied.Experiment results indicate that the majority of the architectural differences do have a significant effect on the agents\u27 performance with respect to the performance measures used in this research. The results of these competitions indicate that while each architectural difference significantly affected its agent\u27s performance, no single architectural difference could be credited as causing XCS\u27s demonstrated superiority in evolving optimal populations. Instead, the data suggests that XCS\u27s ability to evolve optimal populations in the multiplexer and IPD problem domains result from the combined and synergistic effects of multiple architectural differences.In addition, it is demonstrated that XCS is able to reliably evolve the Optimal Population [O] against the TFT opponent. This result supports Kovacs\u27 Optimality Hypothesis in the IPD environment and is significant because it is the first demonstrated occurrence of this ability in an environment other than the multiplexer and Woods problem domains.It is therefore apparent that while XCS performs better than its LCS-based counterparts, its demonstrated superiority may not be attributed to a single architectural characteristic. Instead, XCS\u27s ability to evolve optimal classifier populations in the multiplexer problem domain and in the IPD problem domain studied in this research results from the combined and synergistic effects of multiple architectural differences

Incentive Contracts in Multi-agent Systems: Theory and Applications

This thesis studies incentive contracts in multi-agent systems with applications to transportation policy. The early adoption of emerging transportation systems such as electric vehicles (EVs), peer-to-peer ridesharing, and automated vehicles (AVs) relies on governmental incentives. Those incentives help achieve a specific market share target, prevent irregular behaviors, and enhance social benefit. Yet, two challenges may impede the implementation of such incentive policies. First, the government and subsidized organizations must confront the uncertainty in a market; Second, the government has no access to the organizations' private information, and thus their strategies are unknown to it. In the face of these challenges, a command-and-control incentive policy fails. In Chapter 2, we revisit the primary setting in which a government agency incentivizes the OEM for accelerating the widespread adoption of AVs. This work aspires to offset the negative externalities of AVs in the ``dark-age'' of AV deployment. More specifically, this chapter designs AV subsidies to shorten the early AV market penetration period and maximize the total expected efficiency benefits of AVs. It seeks a generic optimal AV subsidy structure, so-called ``two-threshold'' subsidy policy, which is proven to be more efficient than the social-welfare maximization approach. In Chapter 3, we develop a multi-agent incentive contracts model to address the issue of stimulating a group of non-cooperating agents to act in the principal's interest over a planning horizon. We extend the single-agent incentive contract to a multi-agent setting with history-dependent terminal conditions. Our contributions include: (a) Finding sufficient conditions for the existence of optimal multi-agent incentive contracts and conditions under which they form a unique Nash Equilibrium; (b) Showing that the optimal multi-agent incentive contracts can be solved by a Hamilton-Jacobi-Bellman equation with equilibrium constraints; (c) Proposing a backward iterative algorithm to solve the problem. In Chapter 4, we obtain the optimal EV and charging infrastructure subsidies through the multi-agent incentive contracts model. Widespread adoption of Electric Vehicles (EV) mostly depends on governmental subsidies during the early stage of deployment. The governmental incentives must strike a balance between an EV manufacturer and a charging infrastructure installer. Yet, the current supply of charging infrastructure is not nearly enough to support EV growth over the next decades. We model the joint subsidy problem as a two-agent incentive contract. The government observes two correlated processes -- the EV market penetration and the charging infrastructure expansion. It looks for an optimal policy that maximizes the cumulative social benefit in the face of uncertainty. In our case study, we find that the optimal dynamic subsidies can achieve 70% of the target EV market share in China by 2025, and also maintains the ratio of charging stations per EV. Chapter 5 ends the thesis with conclusions and promising future research directions. In summary, this thesis provides a new approach to appraise transportation and energy policies against exogenous and endogenous risks.PHDIndustrial & Operations EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/155199/1/luoqi_1.pd

The efficient market hypothesis through the eyes of an artificial technical analyst

The academic literature has been reluctant to accept technical analysis as a rational strategy of traders in financial markets. In practice traders and analysts heavily use technical analysis to make investment decisions. To resolve this incongruence the aim of this study is to translate technical analysis into a rigorous formal framework and to investigate its potential failure or success. To avoid subjectivism we design an Artificial Technical Analyst. The empirical study presents the evidence of past market inefficiencies observed on the Tokyo Stock Exchange. The market can be perceived as inefficient if the technical analyst's transaction costs are below the break-even level derived from technical analysis. (English

Sequences of coalition structures in multi-agent systems applied to disaster response

Die Koalitionsbildung ist ein interessantes Thema im Bereich der Multiagentensysteme aufgrund von Herausforderungen bei der praktischen Anwendung, sowie der Komplexität der Berechnung von Lösungen des Problems. Eine Koalition ist ein kurzlebiger Zusammenschluss von Agenten, die ein gemeinsames Ziel verfolgen. Gleichzeitig bietet die kooperative Spieltheorie mit Koalitionen einen formalen Mechanismus zur Analyse von Gruppen aus verschiedenen Akteuren. Daher wird das Problem als Characteristic-Function Game (CFG) modelliert. Dessen Ergebnis sind Aufteilungen einer Menge von Agenten in Koalitionen, sogenannte Koalitionsstrukturen. Allerdings lassen sich nicht alle praktisch auftretenden Probleme effizient mit einer einzigen Koalitionsstruktur lösen. Beispielsweise kann es erforderlich sein, eine Hierarchie von Gruppen zu bilden, in der dann eine Koalitionsstruktur pro Ebene benötigt wird. In der vorliegenden Arbeit werden voneinander abhängige Probleme der Koalitionsbildung untersucht. Insbesondere wird der Schwerpunkt auf die gegenseitige Abhängigkeit von Lösungen (also Koalitionsstrukturen), die aus individuellen Spielen resultieren, gelegt. Angesichts des Mangels an wissenschaftlichen Arbeiten zu diesem Thema wird das Sequential Characteristic-Function Game (SCFG) vorgeschlagen, um die Beziehung zwischen aufeinanderfolgenden Koalitionsstrukturen als Folge von CFGs zu modellieren. Dieses neue Spiel wird erweitert, um spezifische Beschränkungen für jedes CFG in der Spielsequenz zu ermöglichen. Darüber hinaus wird gezeigt, dass das zugrunde liegende SCFG-Problem PSPACE-vollständig ist. Es werden ein exakter Algorithmus zur Berechnung von Lösungen von SCFG-Instanzen, sowie zwei heuristische Algorithmen vorgeschlagen. Die letzte Herausforderung der vorliegenden Arbeit ist die Modellierung eines Katastrophenhilfseinsatzes, bei dem das Einsatzleitsystem (engl. Incident Command System) verwendet wird, mithilfe der vorgeschlagenen Techniken und Algorithmen.Coalition formation has long been an interesting topic of research in Multi-Agent Systems, either for its practical applications or complexity issues. A coalition is commonly understood as a short-lived and goal-directed structure, in which the agents join forces to achieve a goal. Cooperative game theory has been used as a formal mechanism to analyse the problem of grouping agents into coalitions. The problem is then modelled by a Characteristic-Function Game (CFG) in which the outcome is a coalition structure: a partition of agents into coalitions. However, not all problems can be efficiently solved using a single coalition structure. For instance, one might be interested in a group hierarchy in which a coalition structure per level is required. In this thesis, we investigate coalition formation problems that are interdependent. In particular, we focus on the interdependence among solutions (i.e., coalition structures) produced by each game individually. Given the lack of work on this topic, we propose a novel game named Sequential Characteristic-Function Game (SCFG), which aims to model the relationships between subsequent coalition structures in a sequence of CFGs. We approach the resulting problem under both theoretical and practical perspectives. We extend the proposed game to allow fine-grained constraints being induced over each CFG in the sequence. Also, we show that the underlying SCFG problem is PSPACE-complete. From an algorithmic viewpoint, we propose an exact algorithm based on dynamic programming, as well as two heuristic algorithms to compute solutions for SCFG instances. We show that there exists a trade-off in choosing one algorithm over the others. Moreover, we model a disaster response operation that employs the incident command system framework, and we show how one can apply our proposed framework and algorithms to solve such an interesting problem