611,531 research outputs found
Near Optimal Exploration-Exploitation in Non-Communicating Markov Decision Processes
While designing the state space of an MDP, it is common to include states
that are transient or not reachable by any policy (e.g., in mountain car, the
product space of speed and position contains configurations that are not
physically reachable). This leads to defining weakly-communicating or
multi-chain MDPs. In this paper, we introduce \tucrl, the first algorithm able
to perform efficient exploration-exploitation in any finite Markov Decision
Process (MDP) without requiring any form of prior knowledge. In particular, for
any MDP with communicating states, actions and
possible communicating next states,
we derive a regret bound, where is the diameter
(i.e., the longest shortest path) of the communicating part of the MDP. This is
in contrast with optimistic algorithms (e.g., UCRL, Optimistic PSRL) that
suffer linear regret in weakly-communicating MDPs, as well as posterior
sampling or regularised algorithms (e.g., REGAL), which require prior knowledge
on the bias span of the optimal policy to bias the exploration to achieve
sub-linear regret. We also prove that in weakly-communicating MDPs, no
algorithm can ever achieve a logarithmic growth of the regret without first
suffering a linear regret for a number of steps that is exponential in the
parameters of the MDP. Finally, we report numerical simulations supporting our
theoretical findings and showing how TUCRL overcomes the limitations of the
state-of-the-art
Knowledge-intensive software design systems: Can too much knowledge be a burden?
While acknowledging the considerable benefits of domain-specific, knowledge-intensive approaches to automated software engineering, it is prudent to carefully examine the costs of such approaches, as well. In adding domain knowledge to a system, a developer makes a commitment to understanding, representing, maintaining, and communicating that knowledge. This substantial overhead is not generally associated with domain-independent approaches. In this paper, I examine the downside of incorporating additional knowledge, and illustrate with examples based on our experience in building the SIGMA system. I also offer some guidelines for developers building domain-specific systems
Recommended from our members
Performance analysis of a message-oriented knowledge-base
First-order Horn logic is a useful formalism to design knowledge-based systems. When implemented on a sequential von Neumann computer, the main limitation of such systems is performance. We present a message-driven model for function-free Horn logic, where the knowledge base is represented as a network of logical processing elements communicating with one another exclusively through messages. The lack of centralized control and centralized memory makes this model suitable to implementation on a highly-parallel asynchronous computer architecture.The primary contribution of this paper is a performance analysis of this message-driven system and a comparison with a sequential resolution scheme using backtracking. For both approaches, closed form expressions for the performance results are derived and compared
The Role of Middle Range Publications in the Development of Engineering Knowledge
This paper explores the role of publications in the development of engineering knowledge. Previous studies of scientific and technical publications tend to assume that engineers are like scientists in their use of scientific journals as a means of communicating new technical knowledge. But science differs from technology and we should not expect scientists and engineers to use the same sources of knowledge. We contend that previous studies of publications have been flawed because they ignore other forms of publication more suited to the communication of technical and engineering knowledge. This paper argues that technologists use "middle range" publications to exchange knowledge and explore implications of their technological experiences. By providing more visual images, experience-based reports and background information on technologies and products, middle range publications better reflect the ways in which engineers think and work. They allow for visual conversations and support visual communities. The paper provides a detailed exploration of the role of middle range publications and suggests a framework for future research on patterns of publication by technologists and engineers.engineering knowledge, engineering and design organisations, construction, scientific publications, technical publications, innovation studies
Recommended from our members
Commune Health Workers' Methadone Maintenance Treatment (MMT) Knowledge and Perceived Difficulties Providing Decentralized MMT Services in Vietnam.
BackgroundWith the initial establishment of countrywide methadone maintenance therapy (MMT) system, Vietnam is in the process of expanding and decentralizing the MMT program to community-based healthcare settings.ObjectiveThe study aimed to measure the MMT-related knowledge and perceived difficulties in treating patient who use drugs (PWUD) among community-based healthcare providers, e.g., commune health workers (CHW), and examine its correlated factors.MethodsA total of 300 CHW from 60 communes in two provinces of Vietnam completed a survey using Audio Computer-Assisted Self-Interview (ACASI) method. Twelve true-or-false questions were used to assess the CHW's MMT-related knowledge. The CHW's background characteristics and perceived difficulties treating PWUD were recorded.ResultsThe mean MMT knowledge score was 8.2 (SD = 1.2; range: 5-11). Misconceptions toward the benefits, procedure, and side effects of MMT were prevalent. The participants perceived varying degrees of difficulties in recruiting, engaging, and communicating with PWUD. With all covariates holding constant, younger age (standardized įŗ = -0.166; p = 0.0078) was associated with less MMT-related knowledge. Number of PWUD seen in a month and MMT-related knowledge was associated with less perceived difficulties treating PWUD. Conclusions/importance: The finding shed lights on the CHW's knowledge gap, which need to be addressed to facilitate the decentralization of MMT services in Vietnam. In preparation for a decentralized MMT service delivery model, specially designed training is warranted to equip CHW with knowledge and confidence to provide MMT-related services to PWUD
Symmetric and Synchronous Communication in Peer-to-Peer Networks
Motivated by distributed implementations of game-theoretical algorithms, we
study symmetric process systems and the problem of attaining common knowledge
between processes. We formalize our setting by defining a notion of
peer-to-peer networks(*) and appropriate symmetry concepts in the context of
Communicating Sequential Processes (CSP), due to the common knowledge creating
effects of its synchronous communication primitives. We then prove that CSP
with input and output guards makes common knowledge in symmetric peer-to-peer
networks possible, but not the restricted version which disallows output
statements in guards and is commonly implemented.
(*) Please note that we are not dealing with fashionable incarnations such as
file-sharing networks, but merely use this name for a mathematical notion of a
network consisting of directly connected peers "treated on an equal footing",
i.e. not having a client-server structure or otherwise pre-determined roles.)Comment: polished, modernized references; incorporated referee feedback from
MPC'0
Developing the formal structures of artistic practice-as-research
In this article I discuss a topic that is emerging as a valuable paradigm for creative practitioners - practice-as-research. There is some controversy over this term that, I believe, goes to the heart of our understanding of the nature of knowledge. The controversy relates to the idea that practice and research are two inherently different types of activity and therefore that it impossible to engage in one āasā the other. Tim Ingoldās (2011) work on the anthropology of knowledge and skill alongside a broader stream of work on cognition and perception (see for example Lakoff & Johnson 2003 and Gibson 1979) suggests that both artistic practice and academic research involve āpuzzle-solvingā¦ carried on within the context of involvement in a real world of persons, objects and relations.ā (Ingold 2011, p.419). The argument revolves around the notion that there is no such thing as disembodied or abstract knowledge and that all knowledge is both embodied and personally related to the world one inhabits. As such, the written word provides a schematic system for representing the much richer communication processes of speech and bodily experience. The written word, however, can only be understood through reference to our lived experience. Lave (1990, p.310) has termed this āunderstanding in practiceā as a knowledge ābased on rich expectations generated over time about its shapeā (Lave 1990, p.323). Scholarly research outputs and their modes of publication are still firmly entrenched in the printed word. I will explore strategies for communicating the non-verbal knowledge that forms the basis of much practice-as-research
Beware of the āLessonsā you already learned
We live in a society of knowledge. Knowledge is the fundamental key to hold the power of the things that surround us. Without knowledge, we are not able to access many activities that we take for granted, such as driving a car or communicating through a smartphone.
Knowledge is everywhere. It is inside us and it regulates, partially, what we do and donāt do. But not all knowledge we have is true and useful. Some of that knowledge are ālessonsā that arenāt in our best interest. We, as individuals or society, sometimes learnālessonsā that arenāt good for us.info:eu-repo/semantics/publishedVersio
- ā¦