Perseus: Randomized Point-based Value Iteration for POMDPs

Partially observable Markov decision processes (POMDPs) form an attractive and principled framework for agent planning under uncertainty. Point-based approximate techniques for POMDPs compute a policy based on a finite set of points collected in advance from the agents belief space. We present a randomized point-based value iteration algorithm called Perseus. The algorithm performs approximate value backup stages, ensuring that in each backup stage the value of each point in the belief set is improved; the key observation is that a single backup may improve the value of many belief points. Contrary to other point-based methods, Perseus backs up only a (randomly selected) subset of points in the belief set, sufficient for improving the value of each belief point in the set. We show how the same idea can be extended to dealing with continuous action spaces. Experimental results show the potential of Perseus in large scale POMDP problems

Immune Responses of Chronic HCV Infection

Abstract The hepatitis B virus (HBV) is one of the smallest enveloped double-stranded DNA viruses, belonging to the family of hepadnaviridae. HBV replicates via an RNA intermediate and causes both acute and chronic hepatitis. The Australia antigen, now known as the hepatitis B surface antigen (HBsAg) – which is the hallmark of a chronic hepatitis B (CHB) infection – was discovered by Dr. Baruch Samuel Blumberg, for which he received the Nobel prize in Medicine in 1976. Approximately 2 billion people worldwide have been in contact with HBV. Of these, circa 240 – 400 million people are chronically infected. The prevalence of CHB infection is high (> 8%) in resource poor countries, such as in East Asia, sub-Saharan Africa, and in the Amazon, and intermediate (2 – 7%) in the Mediterranean basin, Eastern Europe and Russia. The prevalence in Western countries is low (less than 2%). HBV can survive outside the body for at least 7 days. During this period, it may still cause an acute or chronic infection to people who are not protected by vaccination. HBV is transmitted by percutaneous and mucous membrane exposure to infected body fluids such as serum, saliva and semen. The major route of transmission is thought to be vertical, especially in high-endemic areas. Also horizontal infections occur during early childhood. Among adults, high-risk sexual behavior is the most important risk factor for HBV infection. HBV is predominantly found in the liver as it has a strong preference for infecting hepatocytes. It is thought that HBV is not directly cytopathic, while the proteins produced by the virus and the viral mini-chromosome – also known as covalently closed circular DNA (cccDNA) – may have carcinogenic effects. Moreover, the host immune response directed at infected hepatocytes leads to inflammation of the liver, which results in liver damage, fibrosis, and the development of liver cirrhosis, decompensation and hepatocellular carcinoma (HCC). Overall, 30% of cirrhosis and 53% of all HCC is attributable to CHB infection. Worldwide, approximately 780.000 patients die each year due to an HBV infection: 650.000 due to the sequelae of CHB (cirrhosis, HCC) and another 130.000 from acute HBV

The Beam Conditions Monitor of the LHCb Experiment

The LHCb experiment at the European Organization for Nuclear Research (CERN) is dedicated to precision measurements of CP violation and rare decays of B hadrons. Its most sensitive components are protected by means of a Beam Conditions Monitor (BCM), based on polycrystalline CVD diamond sensors. Its configuration, operation and decision logics to issue or remove the beam permit signal for the Large Hadron Collider (LHC) are described in this paper.Comment: Index Terms: Accelerator measurement systems, CVD, Diamond, Radiation detector

Solving Transition-Independent Multi-agent MDPs with Sparse Interactions (Extended version)

In cooperative multi-agent sequential decision making under uncertainty, agents must coordinate to find an optimal joint policy that maximises joint value. Typical algorithms exploit additive structure in the value function, but in the fully-observable multi-agent MDP setting (MMDP) such structure is not present. We propose a new optimal solver for transition-independent MMDPs, in which agents can only affect their own state but their reward depends on joint transitions. We represent these dependencies compactly in conditional return graphs (CRGs). Using CRGs the value of a joint policy and the bounds on partially specified joint policies can be efficiently computed. We propose CoRe, a novel branch-and-bound policy search algorithm building on CRGs. CoRe typically requires less runtime than the available alternatives and finds solutions to problems previously unsolvable

Error blindness and motivational significance: Shifts in networks centering on anterior insula co-vary with error awareness and pupil dilation

This investigation aims to further our understanding of the brain mechanisms underlying the awareness of one's erroneous actions. While all errors are registered as such in the rostral cingulate zone, errors enter awareness only when the anterior insula cortex is activated. Aware but not unaware errors elicit autonomic nervous system reactivity. Our aim is to investigate the hypothesis that activation in the insula during error awareness is related to autonomic arousal and to inter-regional interactions with other areas of the brain. To examine the role of the anterior insula in error awareness, we assessed its functional connectivity to other brain regions along with autonomic nervous system reactivity in young healthy participants who underwent simultaneous pupil-diameter and functional magnetic resonance imaging measurements while performing a complex and error-prone task. Error blindness was associated with failures to engage sufficient autonomic reactivity. During aware errors increased pupil-diameter along with increased task-related activation within, and increased connectivity between anterior insula and task-related networks suggested an increased capacity for action-control information transfer. Increased pupil-diameter during aware errors was furthermore associated with decreased activation of the default-mode network along with decreased insular connectivity with regions of the default mode system, possibly reflecting decreased task-irrelevant information processing. This shifting mechanism may be relevant to a better understanding of how the brain and the autonomic nervous system interact to enable efficient adaptive behavior during cognitive challenge