Safe Reinforcement Learning Using Formally Verified Abstract Policies

Reinforcement learning (RL) is an artificial intelligence technique for finding optimal solutions for sequential decision-making problems modelled as Markov decision processes (MDPs). Objectives are represented as numerical rewards in the model where positive values represent achievements and negative values represent failures. An autonomous agent explores the model to locate rewards with the goal to learn behaviour which will cumulate the largest reward possible. Despite RL successes in applications ranging from robotics and planning systems to sensing, it has so far had little appeal in mission- and safety-critical systems where unpredictable agent actions could lead to mission failure, risks to humans, itself or other systems, or violations of legal requirements. This is due to the difficulty of encoding non-trivial requirements of agent behaviour through rewards alone. This thesis introduces assured reinforcement learning (ARL), a safe RL approach that restricts agent actions, during and after learning. This restriction is based on formally verified policies synthesised for a high-level, abstract MDP that models the safety-relevant aspects of the RL problem. The resulting actions form overall solutions whose properties satisfy strict safety and optimality requirements. Next, ARL with knowledge revision is introduced, allowing ARL to still be used if the initial knowledge for generating action constraints proves to be incorrect. Additionally, two case studies are introduced to test the efficacy of ARL: the first is an adaptation of the benchmark flag collection navigation task and the second is an assisted-living planning system. Finally, an architecture for runtime ARL is proposed to allow ARL to be utilised in real-time systems. ARL is empirically evaluated and is shown to successfully satisfy strict safety and optimality requirements and, furthermore, with knowledge revision and action reuse, it can be successfully applied in environments where initial information may prove incomplete or incorrect

Study on architecture of self-adaptive software to changing environments

制度:新 ; 文部省報告番号:乙2110号 ; 学位の種類:博士(工学) ; 授与年月日:2007/7/26 ; 早大学位記番号:新460

Semantic multimedia modelling & interpretation for annotation

The emergence of multimedia enabled devices, particularly the incorporation of cameras in mobile phones, and the accelerated revolutions in the low cost storage devices, boosts the multimedia data production rate drastically. Witnessing such an iniquitousness of digital images and videos, the research community has been projecting the issue of its significant utilization and management. Stored in monumental multimedia corpora, digital data need to be retrieved and organized in an intelligent way, leaning on the rich semantics involved. The utilization of these image and video collections demands proficient image and video annotation and retrieval techniques. Recently, the multimedia research community is progressively veering its emphasis to the personalization of these media. The main impediment in the image and video analysis is the semantic gap, which is the discrepancy among a user’s high-level interpretation of an image and the video and the low level computational interpretation of it. Content-based image and video annotation systems are remarkably susceptible to the semantic gap due to their reliance on low-level visual features for delineating semantically rich image and video contents. However, the fact is that the visual similarity is not semantic similarity, so there is a demand to break through this dilemma through an alternative way. The semantic gap can be narrowed by counting high-level and user-generated information in the annotation. High-level descriptions of images and or videos are more proficient of capturing the semantic meaning of multimedia content, but it is not always applicable to collect this information. It is commonly agreed that the problem of high level semantic annotation of multimedia is still far from being answered. This dissertation puts forward approaches for intelligent multimedia semantic extraction for high level annotation. This dissertation intends to bridge the gap between the visual features and semantics. It proposes a framework for annotation enhancement and refinement for the object/concept annotated images and videos datasets. The entire theme is to first purify the datasets from noisy keyword and then expand the concepts lexically and commonsensical to fill the vocabulary and lexical gap to achieve high level semantics for the corpus. This dissertation also explored a novel approach for high level semantic (HLS) propagation through the images corpora. The HLS propagation takes the advantages of the semantic intensity (SI), which is the concept dominancy factor in the image and annotation based semantic similarity of the images. As we are aware of the fact that the image is the combination of various concepts and among the list of concepts some of them are more dominant then the other, while semantic similarity of the images are based on the SI and concept semantic similarity among the pair of images. Moreover, the HLS exploits the clustering techniques to group similar images, where a single effort of the human experts to assign high level semantic to a randomly selected image and propagate to other images through clustering. The investigation has been made on the LabelMe image and LabelMe video dataset. Experiments exhibit that the proposed approaches perform a noticeable improvement towards bridging the semantic gap and reveal that our proposed system outperforms the traditional systems

Normativity and Aristotelian virtue ethics: an evaluation and reconciliation

In recent decades, Aristotelian virtue ethics has reemerged as an alternative to deduction-based moral theories. Yet, Aristotelian virtue ethics has often been conceived by its proponents as well as its detractors, as an approach to ethical thinking that is neither normative in nature nor capable of being formulated in normative terms. In this thesis, I argue that the fundamental elements of Aristotelian virtue ethics, examined and modified in light of modern thinking, provide the basis for a systematized, normative ethical theory. I further argue that such a theory can be grounded in induction, rather than deduction, and that it can fully acknowledge and incorporate the ethical significance of particulars, particular relationships, and human experience. I suggest that an induction-informed normative theory not only avoids such logical pitfalls as Hume's "is-ought" objection and concerns pertaining to the truth-value of moral claims, but also that it provides an accurate account of our moral and non-moral experience, as well as of their areas of intersection. I propose methods for evaluating the acceptability of general guidelines and singular moral judgements, and I argue that these methods can be successfully achieved within, and enhanced by, the framework of Aristotelian virtue ethics. I examine various aspects of moral theory in general and Aristotelian virtue ethics in particular (e.g. principles and guidelines, human nature and telos, virtue, partially and universalizability), and argue for their place within and relationship to an induction-informed normative moral theory. I reply to criticisms levelled against Aristotelian ethical theory and, in so doing, argue that Aristotle's classification of arete as a dunamis in the Rhetoric has significant implications for moral theory, argue for the claims and obligations generated by particular relationships, and reevaluate the role of the phronimos. I review the logical and practical implications of an inductive model, and suggest not only that such a model is more consistent and more practicable than are current deduction-based normative theories, but also that it calls into question our standard conceptualization of normativity. In closing, I suggest a reexamination of "normativity" in terms of the function of normative theory

ON JUDGEMENT: PSYCHOLOGICAL GENESIS, INTENTIONALITY AND GRAMMAR

This thesis explores conceptions of judgement which have been central to various philosophical and scientific traditions. Beginning with Hume, I situate his conception of judgement within his overarching constructivist program, his science of man. Defending Hume from criticism regarding the naturalistic credentials of this program, I argue that Hume’s science of man, along with the conception of judgement which is integral to it, is appropriately understood as a forerunner to contemporary cognitive science. Despite this, I contend that Hume’s conception of judgement prompts a problem regarding the intentionality of judgement – a problem which he does not adequately address. In the second part of my thesis I show how the intentionality problem which Hume grapples with is also crucial, constituting a point of departure, for Kant’s transcendental undertaking. Following Kant’s reasoning, I illustrate how an original concern with this intentionality issue leads Kant to a distinct conception of judgement, according to which concepts only exist in the context of a judgement. Having arrived at Kant’s conception of a judgement, the remainder of the thesis is devoted to the issue of judgement forms. Kant’s postulation of these forms is closely related to his conception of judgement, and I seek to establish both how these forms ought to be understood and how they might be derived. In relation to this latter issue, I suggest that there may a role for contemporary work in Generative Grammar. Specifically, I suggest that it may be viable to understand the forms of judgement as grammatical in nature, thereby securing an interdisciplinary connection between a philosophy of judgement and the empirical investigation of grammar

Semantic multimedia modelling & interpretation for search & retrieval

With the axiomatic revolutionary in the multimedia equip devices, culminated in the proverbial proliferation of the image and video data. Owing to this omnipresence and progression, these data become the part of our daily life. This devastating data production rate accompanies with a predicament of surpassing our potentials for acquiring this data. Perhaps one of the utmost prevailing problems of this digital era is an information plethora. Until now, progressions in image and video retrieval research reached restrained success owed to its interpretation of an image and video in terms of primitive features. Humans generally access multimedia assets in terms of semantic concepts. The retrieval of digital images and videos is impeded by the semantic gap. The semantic gap is the discrepancy between a user’s high-level interpretation of an image and the information that can be extracted from an image’s physical properties. Content- based image and video retrieval systems are explicitly assailable to the semantic gap due to their dependence on low-level visual features for describing image and content. The semantic gap can be narrowed by including high-level features. High-level descriptions of images and videos are more proficient of apprehending the semantic meaning of image and video content. It is generally understood that the problem of image and video retrieval is still far from being solved. This thesis proposes an approach for intelligent multimedia semantic extraction for search and retrieval. This thesis intends to bridge the gap between the visual features and semantics. This thesis proposes a Semantic query Interpreter for the images and the videos. The proposed Semantic Query Interpreter will select the pertinent terms from the user query and analyse it lexically and semantically. The proposed SQI reduces the semantic as well as the vocabulary gap between the users and the machine. This thesis also explored a novel ranking strategy for image search and retrieval. SemRank is the novel system that will incorporate the Semantic Intensity (SI) in exploring the semantic relevancy between the user query and the available data. The novel Semantic Intensity captures the concept dominancy factor of an image. As we are aware of the fact that the image is the combination of various concepts and among the list of concepts some of them are more dominant then the other. The SemRank will rank the retrieved images on the basis of Semantic Intensity. The investigations are made on the LabelMe image and LabelMe video dataset. Experiments show that the proposed approach is successful in bridging the semantic gap. The experiments reveal that our proposed system outperforms the traditional image retrieval systems

Paradoxes of interactivity: perspectives for media theory, human-computer interaction, and artistic investigations

Current findings from anthropology, genetics, prehistory, cognitive and neuroscience indicate that human nature is grounded in a co-evolution of tool use, symbolic communication, social interaction and cultural transmission. Digital information technology has recently entered as a new tool in this co-evolution, and will probably have the strongest impact on shaping the human mind in the near future. A common effort from the humanities, the sciences, art and technology is necessary to understand this ongoing co- evolutionary process. Interactivity is a key for understanding the new relationships formed by humans with social robots as well as interactive environments and wearables underlying this process. Of special importance for understanding interactivity are human-computer and human-robot interaction, as well as media theory and New Media Art. "Paradoxes of Interactivity" brings together reflections on "interactivity" from different theoretical perspectives, the interplay of science and art, and recent technological developments for artistic applications, especially in the realm of sound

Paradoxes of Interactivity

Current findings from anthropology, genetics, prehistory, cognitive and neuroscience indicate that human nature is grounded in a co-evolution of tool use, symbolic communication, social interaction and cultural transmission. Digital information technology has recently entered as a new tool in this co-evolution, and will probably have the strongest impact on shaping the human mind in the near future. A common effort from the humanities, the sciences, art and technology is necessary to understand this ongoing co- evolutionary process. Interactivity is a key for understanding the new relationships formed by humans with social robots as well as interactive environments and wearables underlying this process. Of special importance for understanding interactivity are human-computer and human-robot interaction, as well as media theory and New Media Art. »Paradoxes of Interactivity« brings together reflections on »interactivity« from different theoretical perspectives, the interplay of science and art, and recent technological developments for artistic applications, especially in the realm of sound