A NEW APPROACH TO THE RULE-BASED SYSTEMS DESIGN AND IMPLEMENTATION PROCESS

The paper discusses selected problems encountered in practical rule-based systems (RBS) design and implementation. To solve them XTT, a new visual knowledge representation is introduced. Then a complete, integrated RBS design, implementation and analysis methodology is presented. This methodology is supported by a visual CASE tool called Mirella.The main goal is to move the design procedure to a more abstract, logical level, where knowledge specification is based on use of abstract rule representation. The design specification is automatically translated into Prolog code, so the designer can focus on logical specification of safety and reliability. On the other hand, system formal aspects are automatically verified on-line during the design, so that its verifiable characteristics are preserved

Semantic Business Process Modeling

This book presents a process-oriented business modeling framework based on semantic technologies. The framework consists of modeling languages, methods, and tools that allow for semantic modeling of business motivation, business policies and rules, and business processes. Quality of the proposed modeling framework is evaluated based on the modeling content of SAP Solution Composer and several real-world business scenarios

Exploring Bandit Algorithms for Automatic Content Selection

Multi-armed bandit (MAB) problem is derived from slot machines in the casino. It is about how a gambler could pull the arms in order to maximize total reward. In this sense, the gambler needs to decide which arm to explore in order to gain more knowledge, and which arm to exploit in order to guarantee the total payoff. This problem is also very common in real world, such as automatic content selection. The website is like a gambler. It needs to select proper content to recommend to the visitors, trying to maximize click through rate (CTR). Bandit algorithms are very suitable for this kind of issue. Because they are able to deal with exploration and exploitation trade-off with high churning data. When context is considered during content selection, we model it as contextual bandit problems. In this thesis, we evaluate several popular bandit algorithms in different bandit settings. And we propose our own approach to solve a real world automatic content selection case. Our experiments demonstrate that bandit algorithms are efficient in automatic content selection

Return predictability and its implications for portfolio selection

This thesis inquires into a range of issues in return predictability and its implications. First, the thesis investigates estimation bias in predictive regressions. This research stresses the importance of accounting for the bias when studying predictability. To tackle the problem of biased estimation, a general and convenient method based on the jackknife technique is proposed. The proposed method reduces the bias for both single- and multiple-regressor models and for both short- and long-horizon regressions. Compared with the existing bias-reduction methods in the literature, the proposed method is more stable, robust and flexible. More importantly, it can successfully reduce the estimation bias in long-horizon regressions, whereas the existing bias-reduction methods in the literature cease to work. The effectiveness of the proposed method is demonstrated by simulations and empirical estimates of common predictive models in finance. Empirical results show that the significant predictive variables under ordinary least squares become insignificant after adjusting for the finite-sample bias. These results cast doubt on conclusions drawn in earlier studies on the return predictability by these variables. Next, this thesis examines the predictability of return distributions. It provides detailed insights into predictability of the entire stock and bond return distributions in a quantile regression framework. The difficulty experienced in establishing predictability of the conditional mean through lagged predictor variables does not imply that other parts of the return distribution cannot be predicted. Indeed, many variables are found to have significant but heterogenous effects on the return distributions of stocks and bonds. The thesis establishes a quantile-copula framework for modelling conditional joint return distributions. This framework hinges on quantile regression for marginal return distributions and a copula for the return dependence structure. The framework is shown to be flexible and general enough to model a joint distribution while, at the same time, capturing any non-Gaussian characteristics in both marginal and joint returns. The thesis then explores the implications of return distribution predictability for portfolio selection. A distribution-based framework for portfolio selection is developed which consists of the joint return distribution modelled by the quantile-copula approach and an objective function accommodating higher-order moments. Threshold-accepting optimisation technique is used for obtaining optimal allocation weights. This proposed framework extends traditional moment-based portfolio selection in order to utilise the whole predicted return distribution. The last part of the thesis studies nonlinear dynamics of cross-sectional stock returns using classification and regression trees (CART). The CART models are demonstrated to be a valuable alternative to linear regression analysis in identifying primary drivers of the stock returns. Moreover, a novel hybrid approach combining CART and logistic regression is proposed. This hybrid approach takes advantage of the strengths in both CART and linear parametric models. An empirical application to cross-sectional stock return prediction shows that the hybrid approach captures return dynamics better than either a standalone CART or a logistic model

Automation and Control

Advances in automation and control today cover many areas of technology where human input is minimized. This book discusses numerous types and applications of automation and control. Chapters address topics such as building information modeling (BIM)–based automated code compliance checking (ACCC), control algorithms useful for military operations and video games, rescue competitions using unmanned aerial-ground robots, and stochastic control systems

Many-agent Reinforcement Learning

Multi-agent reinforcement learning (RL) solves the problem of how each agent should behave optimally in a stochastic environment in which multiple agents are learning simultaneously. It is an interdisciplinary domain with a long history that lies in the joint area of psychology, control theory, game theory, reinforcement learning, and deep learning. Following the remarkable success of the AlphaGO series in single-agent RL, 2019 was a booming year that witnessed significant advances in multi-agent RL techniques; impressive breakthroughs have been made on developing AIs that outperform humans on many challenging tasks, especially multi-player video games. Nonetheless, one of the key challenges of multi-agent RL techniques is the scalability; it is still non-trivial to design efficient learning algorithms that can solve tasks including far more than two agents ($N \gg 2$), which I name by \emph{many-agent reinforcement learning} (MARL\footnote{I use the world of ``MARL" to denote multi-agent reinforcement learning with a particular focus on the cases of many agents; otherwise, it is denoted as ``Multi-Agent RL" by default.}) problems. In this thesis, I contribute to tackling MARL problems from four aspects. Firstly, I offer a self-contained overview of multi-agent RL techniques from a game-theoretical perspective. This overview fills the research gap that most of the existing work either fails to cover the recent advances since 2010 or does not pay adequate attention to game theory, which I believe is the cornerstone to solving many-agent learning problems. Secondly, I develop a tractable policy evaluation algorithm -- $\alpha^\alpha$-Rank -- in many-agent systems. The critical advantage of $\alpha^\alpha$-Rank is that it can compute the solution concept of $\alpha$-Rank tractably in multi-player general-sum games with no need to store the entire pay-off matrix. This is in contrast to classic solution concepts such as Nash equilibrium which is known to be $PPAD$-hard in even two-player cases. $\alpha^\alpha$-Rank allows us, for the first time, to practically conduct large-scale multi-agent evaluations. Thirdly, I introduce a scalable policy learning algorithm -- mean-field MARL -- in many-agent systems. The mean-field MARL method takes advantage of the mean-field approximation from physics, and it is the first provably convergent algorithm that tries to break the curse of dimensionality for MARL tasks. With the proposed algorithm, I report the first result of solving the Ising model and multi-agent battle games through a MARL approach. Fourthly, I investigate the many-agent learning problem in open-ended meta-games (i.e., the game of a game in the policy space). Specifically, I focus on modelling the behavioural diversity in meta-games, and developing algorithms that guarantee to enlarge diversity during training. The proposed metric based on determinantal point processes serves as the first mathematically rigorous definition for diversity. Importantly, the diversity-aware learning algorithms beat the existing state-of-the-art game solvers in terms of exploitability by a large margin. On top of the algorithmic developments, I also contribute two real-world applications of MARL techniques. Specifically, I demonstrate the great potential of applying MARL to study the emergent population dynamics in nature, and model diverse and realistic interactions in autonomous driving. Both applications embody the prospect that MARL techniques could achieve huge impacts in the real physical world, outside of purely video games