611,531 research outputs found

    Near Optimal Exploration-Exploitation in Non-Communicating Markov Decision Processes

    Get PDF
    While designing the state space of an MDP, it is common to include states that are transient or not reachable by any policy (e.g., in mountain car, the product space of speed and position contains configurations that are not physically reachable). This leads to defining weakly-communicating or multi-chain MDPs. In this paper, we introduce \tucrl, the first algorithm able to perform efficient exploration-exploitation in any finite Markov Decision Process (MDP) without requiring any form of prior knowledge. In particular, for any MDP with SCS^{\texttt{C}} communicating states, AA actions and Ī“Cā‰¤SC\Gamma^{\texttt{C}} \leq S^{\texttt{C}} possible communicating next states, we derive a O~(DCĪ“CSCAT)\widetilde{O}(D^{\texttt{C}} \sqrt{\Gamma^{\texttt{C}} S^{\texttt{C}} AT}) regret bound, where DCD^{\texttt{C}} is the diameter (i.e., the longest shortest path) of the communicating part of the MDP. This is in contrast with optimistic algorithms (e.g., UCRL, Optimistic PSRL) that suffer linear regret in weakly-communicating MDPs, as well as posterior sampling or regularised algorithms (e.g., REGAL), which require prior knowledge on the bias span of the optimal policy to bias the exploration to achieve sub-linear regret. We also prove that in weakly-communicating MDPs, no algorithm can ever achieve a logarithmic growth of the regret without first suffering a linear regret for a number of steps that is exponential in the parameters of the MDP. Finally, we report numerical simulations supporting our theoretical findings and showing how TUCRL overcomes the limitations of the state-of-the-art

    Knowledge-intensive software design systems: Can too much knowledge be a burden?

    Get PDF
    While acknowledging the considerable benefits of domain-specific, knowledge-intensive approaches to automated software engineering, it is prudent to carefully examine the costs of such approaches, as well. In adding domain knowledge to a system, a developer makes a commitment to understanding, representing, maintaining, and communicating that knowledge. This substantial overhead is not generally associated with domain-independent approaches. In this paper, I examine the downside of incorporating additional knowledge, and illustrate with examples based on our experience in building the SIGMA system. I also offer some guidelines for developers building domain-specific systems

    The Role of Middle Range Publications in the Development of Engineering Knowledge

    Get PDF
    This paper explores the role of publications in the development of engineering knowledge. Previous studies of scientific and technical publications tend to assume that engineers are like scientists in their use of scientific journals as a means of communicating new technical knowledge. But science differs from technology and we should not expect scientists and engineers to use the same sources of knowledge. We contend that previous studies of publications have been flawed because they ignore other forms of publication more suited to the communication of technical and engineering knowledge. This paper argues that technologists use "middle range" publications to exchange knowledge and explore implications of their technological experiences. By providing more visual images, experience-based reports and background information on technologies and products, middle range publications better reflect the ways in which engineers think and work. They allow for visual conversations and support visual communities. The paper provides a detailed exploration of the role of middle range publications and suggests a framework for future research on patterns of publication by technologists and engineers.engineering knowledge, engineering and design organisations, construction, scientific publications, technical publications, innovation studies

    Symmetric and Synchronous Communication in Peer-to-Peer Networks

    Get PDF
    Motivated by distributed implementations of game-theoretical algorithms, we study symmetric process systems and the problem of attaining common knowledge between processes. We formalize our setting by defining a notion of peer-to-peer networks(*) and appropriate symmetry concepts in the context of Communicating Sequential Processes (CSP), due to the common knowledge creating effects of its synchronous communication primitives. We then prove that CSP with input and output guards makes common knowledge in symmetric peer-to-peer networks possible, but not the restricted version which disallows output statements in guards and is commonly implemented. (*) Please note that we are not dealing with fashionable incarnations such as file-sharing networks, but merely use this name for a mathematical notion of a network consisting of directly connected peers "treated on an equal footing", i.e. not having a client-server structure or otherwise pre-determined roles.)Comment: polished, modernized references; incorporated referee feedback from MPC'0

    Developing the formal structures of artistic practice-as-research

    Get PDF
    In this article I discuss a topic that is emerging as a valuable paradigm for creative practitioners - practice-as-research. There is some controversy over this term that, I believe, goes to the heart of our understanding of the nature of knowledge. The controversy relates to the idea that practice and research are two inherently different types of activity and therefore that it impossible to engage in one ā€˜asā€™ the other. Tim Ingoldā€™s (2011) work on the anthropology of knowledge and skill alongside a broader stream of work on cognition and perception (see for example Lakoff & Johnson 2003 and Gibson 1979) suggests that both artistic practice and academic research involve ā€˜puzzle-solvingā€¦ carried on within the context of involvement in a real world of persons, objects and relations.ā€™ (Ingold 2011, p.419). The argument revolves around the notion that there is no such thing as disembodied or abstract knowledge and that all knowledge is both embodied and personally related to the world one inhabits. As such, the written word provides a schematic system for representing the much richer communication processes of speech and bodily experience. The written word, however, can only be understood through reference to our lived experience. Lave (1990, p.310) has termed this ā€˜understanding in practiceā€™ as a knowledge ā€˜based on rich expectations generated over time about its shapeā€™ (Lave 1990, p.323). Scholarly research outputs and their modes of publication are still firmly entrenched in the printed word. I will explore strategies for communicating the non-verbal knowledge that forms the basis of much practice-as-research

    Beware of the ā€œLessonsā€ you already learned

    Get PDF
    We live in a society of knowledge. Knowledge is the fundamental key to hold the power of the things that surround us. Without knowledge, we are not able to access many activities that we take for granted, such as driving a car or communicating through a smartphone. Knowledge is everywhere. It is inside us and it regulates, partially, what we do and donā€™t do. But not all knowledge we have is true and useful. Some of that knowledge are ā€œlessonsā€ that arenā€™t in our best interest. We, as individuals or society, sometimes learnā€œlessonsā€ that arenā€™t good for us.info:eu-repo/semantics/publishedVersio
    • ā€¦
    corecore