Adaptive Load Balancing: A Study in Multi-Agent Learning

We study the process of multi-agent reinforcement learning in the context of load balancing in a distributed system, without use of either central coordination or explicit communication. We first define a precise framework in which to study adaptive load balancing, important features of which are its stochastic nature and the purely local information available to individual agents. Given this framework, we show illuminating results on the interplay between basic adaptive behavior parameters and their effect on system efficiency. We then investigate the properties of adaptive load balancing in heterogeneous populations, and address the issue of exploration vs. exploitation in that context. Finally, we show that naive use of communication may not improve, and might even harm system efficiency.Comment: See http://www.jair.org/ for any accompanying file

Bounded Quantifier Instantiation for Checking Inductive Invariants

We consider the problem of checking whether a proposed invariant $\varphi$ expressed in first-order logic with quantifier alternation is inductive, i.e. preserved by a piece of code. While the problem is undecidable, modern SMT solvers can sometimes solve it automatically. However, they employ powerful quantifier instantiation methods that may diverge, especially when $\varphi$ is not preserved. A notable difficulty arises due to counterexamples of infinite size. This paper studies Bounded-Horizon instantiation, a natural method for guaranteeing the termination of SMT solvers. The method bounds the depth of terms used in the quantifier instantiation process. We show that this method is surprisingly powerful for checking quantified invariants in uninterpreted domains. Furthermore, by producing partial models it can help the user diagnose the case when $\varphi$ is not inductive, especially when the underlying reason is the existence of infinite counterexamples. Our main technical result is that Bounded-Horizon is at least as powerful as instrumentation, which is a manual method to guarantee convergence of the solver by modifying the program so that it admits a purely universal invariant. We show that with a bound of 1 we can simulate a natural class of instrumentations, without the need to modify the code and in a fully automatic way. We also report on a prototype implementation on top of Z3, which we used to verify several examples by Bounded-Horizon of bound 1

Cumulative Incidence Function Estimation Based on Population-Based Biobank Data

Many countries have established population-based biobanks, which are being used increasingly in epidemiolgical and clinical research. These biobanks offer opportunities for large-scale studies addressing questions beyond the scope of traditional clinical trials or cohort studies. However, using biobank data poses new challenges. Typically, biobank data is collected from a study cohort recruited over a defined calendar period, with subjects entering the study at various ages falling between $c_L$ and $c_U$. This work focuses on biobank data with individuals reporting disease-onset age upon recruitment, termed prevalent data, along with individuals initially recruited as healthy, and their disease onset observed during the follow-up period. We propose a novel cumulative incidence function (CIF) estimator that efficiently incorporates prevalent cases, in contrast to existing methods, providing two advantages: (1) increased efficiency, and (2) CIF estimation for ages before the lower limit, $c_L$

Association between sensory impairment and suicidal ideation and attempt: a cross-sectional analysis of nationally representative English household data

OBJECTIVES: Sensory impairments are associated with worse mental health and poorer quality of life, but few studies have investigated whether sensory impairment is associated with suicidal behaviour in a population sample. We investigated whether visual and hearing impairments were associated with suicidal ideation and attempt. DESIGN: National cross-sectional study. SETTING: Households in England. PARTICIPANTS: We analysed data for 7546 household residents in England, aged 16 and over from the 2014 Adult Psychiatric Morbidity Survey. EXPOSURES: Sensory impairment (either visual or hearing), Dual sensory impairment (visual and hearing), visual impairment, hearing impairment. PRIMARY OUTCOME: Suicidal ideation and suicide attempt in the past year. RESULTS: People with visual or hearing sensory impairments had twice the odds of past-year suicidal ideation (OR 2.06; 95% CI 1.17 to 2.73; p<0.001), and over three times the odds of reporting past-year suicide attempt (OR 3.12; 95% CI 1.57 to 6.20; p=0.001) compared with people without these impairments. Similar results were found for hearing and visual impairments separately and co-occurring. CONCLUSIONS: We found evidence that individuals with sensory impairments are more likely to have thought about or attempted suicide in the past year than individuals without

Constraint rule-based programming of norms for electronic institutions

Peer reviewedPostprin

The Emergence of Norms via Contextual Agreements in Open Societies

This paper explores the emergence of norms in agents' societies when agents play multiple -even incompatible- roles in their social contexts simultaneously, and have limited interaction ranges. Specifically, this article proposes two reinforcement learning methods for agents to compute agreements on strategies for using common resources to perform joint tasks. The computation of norms by considering agents' playing multiple roles in their social contexts has not been studied before. To make the problem even more realistic for open societies, we do not assume that agents share knowledge on their common resources. So, they have to compute semantic agreements towards performing their joint actions. %The paper reports on an empirical study of whether and how efficiently societies of agents converge to norms, exploring the proposed social learning processes w.r.t. different society sizes, and the ways agents are connected. The results reported are very encouraging, regarding the speed of the learning process as well as the convergence rate, even in quite complex settings