The Complexity of Graph-Based Reductions for Reachability in Markov Decision Processes

We study the never-worse relation (NWR) for Markov decision processes with an infinite-horizon reachability objective. A state q is never worse than a state p if the maximal probability of reaching the target set of states from p is at most the same value from q, regard- less of the probabilities labelling the transitions. Extremal-probability states, end components, and essential states are all special cases of the equivalence relation induced by the NWR. Using the NWR, states in the same equivalence class can be collapsed. Then, actions leading to sub- optimal states can be removed. We show the natural decision problem associated to computing the NWR is coNP-complete. Finally, we ex- tend a previously known incomplete polynomial-time iterative algorithm to under-approximate the NWR

Planning with Information-Processing Constraints and Model Uncertainty in Markov Decision Processes

Information-theoretic principles for learning and acting have been proposed to solve particular classes of Markov Decision Problems. Mathematically, such approaches are governed by a variational free energy principle and allow solving MDP planning problems with information-processing constraints expressed in terms of a Kullback-Leibler divergence with respect to a reference distribution. Here we consider a generalization of such MDP planners by taking model uncertainty into account. As model uncertainty can also be formalized as an information-processing constraint, we can derive a unified solution from a single generalized variational principle. We provide a generalized value iteration scheme together with a convergence proof. As limit cases, this generalized scheme includes standard value iteration with a known model, Bayesian MDP planning, and robust planning. We demonstrate the benefits of this approach in a grid world simulation.Comment: 16 pages, 3 figure

Spatial Fingerprints of Community Structure in Human Interaction Network for an Extensive Set of Large-Scale Regions

Human interaction networks inferred from country-wide telephone activity recordings were recently used to redraw political maps by projecting their topological partitions into geographical space. The results showed remarkable spatial cohesiveness of the network communities and a significant overlap between the redrawn and the administrative borders. Here we present a similar analysis based on one of the most popular online social networks represented by the ties between more than 5.8 million of its geo-located users. The worldwide coverage of their measured activity allowed us to analyze the large-scale regional subgraphs of entire continents and an extensive set of examples for single countries. We present results for North and South America, Europe and Asia. In our analysis we used the well- established method of modularity clustering after an aggregation of the individual links into a weighted graph connecting equal- area geographical pixels. Our results show fingerprints of both of the opposing forces of dividing local conflicts and of uniting cross-cultural trends of globalization

Glucocorticoids—All-Rounders Tackling the Versatile Players of the Immune System

Glucocorticoids regulate fundamental processes of the human body and control cellular functions such as cell metabolism, growth, differentiation, and apoptosis. Moreover, endogenous glucocorticoids link the endocrine and immune system and ensure the correct function of inflammatory events during tissue repair, regeneration, and pathogen elimination via genomic and rapid non-genomic pathways. Due to their strong immunosuppressive, anti-inflammatory and anti-allergic effects on immune cells, tissues and organs, glucocorticoids significantly improve the quality of life of many patients suffering from diseases caused by a dysregulated immune system. Despite the multitude and seriousness of glucocorticoid-related adverse events including diabetes mellitus, osteoporosis and infections, these agents remain indispensable, representing the most powerful, and cost-effective drugs in the treatment of a wide range of rheumatic diseases. These include rheumatoid arthritis, vasculitis, and connective tissue diseases, as well as many other pathological conditions of the immune system. Depending on the therapeutically affected cell type, glucocorticoid actions strongly vary among different diseases. While immune responses always represent complex reactions involving different cells and cellular processes, specific immune cell populations with key responsibilities driving the pathological mechanisms can be identified for certain autoimmune diseases. In this review, we will focus on the mechanisms of action of glucocorticoids on various leukocyte populations, exemplarily portraying different autoimmune diseases as heterogeneous targets of glucocorticoid actions: (i) Abnormalities in the innate immune response play a crucial role in the initiation and perpetuation of giant cell arteritis (GCA). (ii) Specific types of CD4+ T helper (Th) lymphocytes, namely Th1 and Th17 cells, represent important players in the establishment and course of rheumatoid arthritis (RA), whereas (iii) B cells have emerged as central players in systemic lupus erythematosus (SLE). (iv) Allergic reactions are mainly triggered by several different cytokines released by activated Th2 lymphocytes. Using these examples, we aim to illustrate the versatile modulating effects of glucocorticoids on the immune system. In contrast, in the treatment of lymphoproliferative disorders the pro-apoptotic action of glucocorticoids prevails, but their mechanisms differ depending on the type of cancer. Therefore, we will also give a brief insight into the current knowledge of the mode of glucocorticoid action in oncological treatment focusing on leukemia

Determining utility values in patients with anterior cruciate ligament tears using clinical scoring systems

BACKGROUND: Several instruments and clinical scoring systems have been established to evaluate patients with ligamentous knee injuries. A comparison of individual articles in the literature is challenging, not only because of heterogeneity in methodology, but also due to the variety of the scoring systems used to document clinical outcomes. There is limited information about the correlation between used scores and quality of life with no information being available on the impact of each score on the utility values. The aim of this study was to compare the most commonly used scores for evaluating patients with anterior cruciate ligament (ACL) injuries, and to establish corresponding utility values. These values will be used for the interpretation and comparison of outcome results in the currently available literature for different treatment options. METHODS: Four hypothetical vignettes were defined, based on different levels of activities after rupture of the ACL to simulate typical situations seen in daily practice. A questionnaire, including the Health Utility Index (HUI) for utility values, the IKDC subjective score, the Lysholm and the Tegner score, was created and 25 orthopedic surgeons were asked to fill the questionnaire for each hypothetical patient as proxies for all patients they had treated and who would fit in that hypothetical vignette. RESULTS: The utility value as an indicator for quality of life increased with the level of activity. Having discomforts already during normal activities of daily living was rated with a mean utility value of 0.37 ± 0.19, half of that of a situation where mild sport activity was possible without discomfort (0.78 ± 0.11). All investigated scores were able to distinguish clearly (p < 0.05) between the hypothetical vignettes. However, the utility values correlated best with the IKDC subjective score (r = 0.86, p < 0.001) followed by the Lysholm score (r = 0.77, p < 0.001) and the Tegner score (r = 0.77, p < 0.001). CONCLUSIONS: Here we report the correlation between the most commonly used scores for the assessment of patients with a ruptured ACL and utility values as an indicator of quality of life. Assumptions were based on expert opinions to provide a possible transformation algorithm. The IKDC subjective knee score showed the highest correlation to the quality of life (i.e. HUI) in patients with a ruptured ACL. Confirmation of our results is needed by systematic inclusion of a measurement instrument for utility values in future clinical studies beside the already used clinical knee scoring systems

Impairment of Immunoproteasome Function by β5i/LMP7 Subunit Deficiency Results in Severe Enterovirus Myocarditis

Proteasomes recognize and degrade poly-ubiquitinylated proteins. In infectious disease, cells activated by interferons (IFNs) express three unique catalytic subunits β1i/LMP2, β2i/MECL-1 and β5i/LMP7 forming an alternative proteasome isoform, the immunoproteasome (IP). The in vivo function of IPs in pathogen-induced inflammation is still a matter of controversy. IPs were mainly associated with MHC class I antigen processing. However, recent findings pointed to a more general function of IPs in response to cytokine stress. Here, we report on the role of IPs in acute coxsackievirus B3 (CVB3) myocarditis reflecting one of the most common viral disease entities among young people. Despite identical viral load in both control and IP-deficient mice, IP-deficiency was associated with severe acute heart muscle injury reflected by large foci of inflammatory lesions and severe myocardial tissue damage. Exacerbation of acute heart muscle injury in this host was ascribed to disequilibrium in protein homeostasis in viral heart disease as indicated by the detection of increased proteotoxic stress in cytokine-challenged cardiomyocytes and inflammatory cells from IP-deficient mice. In fact, due to IP-dependent removal of poly-ubiquitinylated protein aggregates in the injured myocardium IPs protected CVB3-challenged mice from oxidant-protein damage. Impaired NFκB activation in IP-deficient cardiomyocytes and inflammatory cells and proteotoxic stress in combination with severe inflammation in CVB3-challenged hearts from IP-deficient mice potentiated apoptotic cell death in this host, thus exacerbating acute tissue damage. Adoptive T cell transfer studies in IP-deficient mice are in agreement with data pointing towards an effective CD8 T cell immune. This study therefore demonstrates that IP formation primarily protects the target organ of CVB3 infection from excessive inflammatory tissue damage in a virus-induced proinflammatory cytokine milieu

Compensatory T Cell Responses in IRG-Deficient Mice Prevent Sustained Chlamydia trachomatis Infections

The obligate intracellular pathogen Chlamydia trachomatis is the most common cause of bacterial sexually transmitted diseases in the United States. In women C. trachomatis can establish persistent genital infections that lead to pelvic inflammatory disease and sterility. In contrast to natural infections in humans, experimentally induced infections with C. trachomatis in mice are rapidly cleared. The cytokine interferon-γ (IFNγ) plays a critical role in the clearance of C. trachomatis infections in mice. Because IFNγ induces an antimicrobial defense system in mice but not in humans that is composed of a large family of Immunity Related GTPases (IRGs), we questioned whether mice deficient in IRG immunity would develop persistent infections with C. trachomatis as observed in human patients. We found that IRG-deficient Irgm1/m3(-/-) mice transiently develop high bacterial burden post intrauterine infection, but subsequently clear the infection more efficiently than wildtype mice. We show that the delayed but highly effective clearance of intrauterine C. trachomatis infections in Irgm1/m3(-/-) mice is dependent on an exacerbated CD4+ T cell response. These findings indicate that the absence of the predominant murine innate effector mechanism restricting C. trachomatis growth inside epithelial cells results in a compensatory adaptive immune response, which is at least in part driven by CD4+ T cells and prevents the establishment of a persistent infection in mice

Reconstruction versus conservative treatment after rupture of the anterior cruciate ligament: cost effectiveness analysis

BACKGROUND: The decision whether to treat conservatively or reconstruct surgically a torn anterior cruciate ligament (ACL) is an ongoing subject of debate. The high prevalence and associated public health burden of torn ACL has led to continuous efforts to determine the best therapeutic approach. A critical evaluation of benefits and expenditures of both treatment options as in a cost effectiveness analysis seems well-suited to provide valuable information for treating physicians and healthcare policymakers. METHODS: A literature review identified four of 7410 searched articles providing sufficient outcome probabilities for the two treatment options for modeling. A transformation key based on the expert opinions of 25 orthopedic surgeons was used to derive utilities from available evidence. The cost data for both treatment strategies were based on average figures compiled by Orthopaedic University Hospital Balgrist and reinforced by Swiss national statistics. A decision tree was constructed to derive the cost-effectiveness of each strategy, which was then tested for robustness using Monte Carlo simulation. RESULTS: Decision tree analysis revealed a cost effectiveness of 16,038 USD/0.78 QALY for ACL reconstruction and 15,466 USD/0.66 QALY for conservative treatment, implying an incremental cost effectiveness of 4,890 USD/QALY for ACL reconstruction. Sensitivity analysis of utilities did not change the trend. CONCLUSION: ACL reconstruction for reestablishment of knee stability seems cost effective in the Swiss setting based on currently available evidence. This, however, should be reinforced with randomized controlled trials comparing the two treatment strategies

Unsupervised record matching with noisy and incomplete data

We consider the problem of duplicate detection in noisy and incomplete data: given a large data set in which each record has multiple entries (attributes), detect which distinct records refer to the same real world entity. This task is complicated by noise (such as misspellings) and missing data, which can lead to records being different, despite referring to the same entity. Our method consists of three main steps: creating a similarity score between records, grouping records together into "unique entities", and refining the groups. We compare various methods for creating similarity scores between noisy records, considering different combinations of string matching, term frequency-inverse document frequency methods, and n-gram techniques. In particular, we introduce a vectorized soft term frequency-inverse document frequency method, with an optional refinement step. We also discuss two methods to deal with missing data in computing similarity scores. We test our method on the Los Angeles Police Department Field Interview Card data set, the Cora Citation Matching data set, and two sets of restaurant review data. The results show that the methods that use words as the basic units are preferable to those that use 3-grams. Moreover, in some (but certainly not all) parameter ranges soft term frequency-inverse document frequency methods can outperform the standard term frequency-inverse document frequency method. The results also confirm that our method for automatically determining the number of groups typically works well in many cases and allows for accurate results in the absence of a priori knowledge of the number of unique entities in the data set