Exploiting Anonymity in Approximate Linear Programming: Scaling to Large Multiagent MDPs (Extended Version)

Many exact and approximate solution methods for Markov Decision Processes (MDPs) attempt to exploit structure in the problem and are based on factorization of the value function. Especially multiagent settings, however, are known to suffer from an exponential increase in value component sizes as interactions become denser, meaning that approximation architectures are restricted in the problem sizes and types they can handle. We present an approach to mitigate this limitation for certain types of multiagent systems, exploiting a property that can be thought of as "anonymous influence" in the factored MDP. Anonymous influence summarizes joint variable effects efficiently whenever the explicit representation of variable identity in the problem can be avoided. We show how representational benefits from anonymity translate into computational efficiencies, both for general variable elimination in a factor graph but in particular also for the approximate linear programming solution to factored MDPs. The latter allows to scale linear programming to factored MDPs that were previously unsolvable. Our results are shown for the control of a stochastic disease process over a densely connected graph with 50 nodes and 25 agents.Comment: Extended version of AAAI 2016 pape

Exploiting feature dynamics for active object recognition

This paper describes a new approach to object recognition for active vision systems that integrates information across multiple observations of an object. The approach exploits the order relationship between successive frames to derive a classifier based on the characteristic motion of local features across visual sweeps. This motion model reveals structural information about the object that can be exploited for recognition. The main contribution of this paper is a recognition system that extends invariant local features (shape contexts) into the time domain by integration of a motion model. Evaluations on one standardized and one custom collected dataset from the humanoid robot in our laboratory demonstrate that the motion model allows higher-quality hypotheses about object categories quicker than a baseline system that treats object views as unordered streams of images

Untersuchungen zur Chemo- und Radiosensitivität und deren Auswirkungen auf Zellzyklus, Apoptoseinduktion und Genexpression in neuroendokrinen Pankreastumorzellen

Die zur Behandlung maligner neuroendokriner gastroenteropankreatischer Tumoren eingesetzten Chemotherapeutika führen bisher zu keiner deutlich verbesserten Überlebenszeit der Patienten. Vor diesem Hintergrund sollte in vitro die Chemosensitivität bzw. Chemoresistenz der neuroendokrinen Pankreastumorzellinie BON I gegenüber verschiedenen Substanzklassen von Chemotherapeutika analysiert und die Veränderungen der Zellzyklusregulation, Apoptoseinduktion und Genexpression charakterisiert werden. Die Mitosehemmstoffe Vinblastin und Vincristin zeigten in äquivalenter Konzentration den stärksten proliferationshemmenden Effekt, den zweitstärksten zeigte Paclitaxel. Paclitaxelbehandlung führte neben Vinblastin zeit- und konzentrationsabhängig zum ausgeprägten Zellzyklusarrest in der G2/M-Phase mit Einleitung der Apoptose. Dagegen erfolgte bei Cisplatin, Doxorubicin, Irinotecan und 5-Aza-2’-Deoxycytidin ein verzögerter schwächerer G2/M-Arrest mit nur geringer Apoptoseinduktion. Vorbehandlung der Zellen mit Paclitaxel und anschließender Bestrahlung führte nur bei hoher Strahlendosis zu einem signifikanten synergistischen Effekt auf die BON I Zellproliferation. BON I Zellen zeigten sich gegenüber 5-Fluorouracil, Carboplatin oder Octreotidacetat nicht sensitiv. Genexpressionsanalysen in Paclitaxel-behandelten versus unbehandelten BON I Zellen konnten die Heraufregulation verschiedener Zykline und Zyklin-assoziierter Kinasen und ihrer Inhibitoren, sowie einiger apoptoseregulierender Gene mit RT-PCR, cDNA-Array und Westernblotanalyse nachweisen. Insgesamt ergaben sich anhand der Ergebnisse keine Hinweise auf eine p53-abhängige oder eine Todesrezeptor-vermittelte Apoptose in Paclitaxel-behandelten BON-1-Zellen. Stattdessen könnte die Paclitaxel-induzierte Heraufregulation von Caspase-9 und eingeschränkt auch von Bax auf einen mitochondrialen Apoptosepathway hinweisen. In dieser Arbeit wurde erstmals ein durch Paclitaxel ausgelöster G2/M-Arrest, die Induktion von Apoptose und ein synergistischer radiosensitivierender Effekt in einer neuroendokrinen Pankreastumorzellinie (Karzinoid) nachgewiesen. Aufgrund der in vitro gewonnenen Ergebnisse wäre daher ein verbessertes klinisches Ansprechen von neuroendokrinen pankreatischen Tumoren (Karzinoid) unter Paclitaxel-Behandlung in vivo denkbar

Exploiting object dynamics for recognition and control

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2007.Includes bibliographical references (p. 127-132).This thesis explores how state-of-the-art object recognition methods can benefit from integrating information across multiple observations of an object. Considered are active vision systems that allow to steer the camera along predetermined trajectories, resulting in sweeps of ordered views of an object. For systems of this kind, a solution is presented that exploits the order relationship between successive frames to derive a classifier based on the characteristic motion of local features across the sweep. It is shown that this motion model reveals structural information about the object that can be exploited for recognition. The main contribution of this thesis is a recognition system that extends invariant local features (shape context) into the time domain by adding the mentioned feature motion model into a joint classifier. Second, an entropy-based view selection scheme is presented that allows the vision system to skip ahead to highly discriminative viewing positions. Using two datasets, one standard (ETH-80) and one collected from our robot head, both feature motion and active view selection extensions are shown to achieve a higher-quality hypothesis about the presented object quicker than a baseline system treating object views as an unordered stream of images.by Philipp Robbel.S.M

Untersuchungen zur Chemo- und Radiosensitivität und deren Auswirkungen auf Zellzyklus, Apoptoseinduktion und Genexpression in neuroendokrinen Pankreastumorzellen

Die zur Behandlung maligner neuroendokriner gastroenteropankreatischer Tumoren eingesetzten Chemotherapeutika führen bisher zu keiner deutlich verbesserten Überlebenszeit der Patienten. Vor diesem Hintergrund sollte in vitro die Chemosensitivität bzw. Chemoresistenz der neuroendokrinen Pankreastumorzellinie BON I gegenüber verschiedenen Substanzklassen von Chemotherapeutika analysiert und die Veränderungen der Zellzyklusregulation, Apoptoseinduktion und Genexpression charakterisiert werden. Die Mitosehemmstoffe Vinblastin und Vincristin zeigten in äquivalenter Konzentration den stärksten proliferationshemmenden Effekt, den zweitstärksten zeigte Paclitaxel. Paclitaxelbehandlung führte neben Vinblastin zeit- und konzentrationsabhängig zum ausgeprägten Zellzyklusarrest in der G2/M-Phase mit Einleitung der Apoptose. Dagegen erfolgte bei Cisplatin, Doxorubicin, Irinotecan und 5-Aza-2’-Deoxycytidin ein verzögerter schwächerer G2/M-Arrest mit nur geringer Apoptoseinduktion. Vorbehandlung der Zellen mit Paclitaxel und anschließender Bestrahlung führte nur bei hoher Strahlendosis zu einem signifikanten synergistischen Effekt auf die BON I Zellproliferation. BON I Zellen zeigten sich gegenüber 5-Fluorouracil, Carboplatin oder Octreotidacetat nicht sensitiv. Genexpressionsanalysen in Paclitaxel-behandelten versus unbehandelten BON I Zellen konnten die Heraufregulation verschiedener Zykline und Zyklin-assoziierter Kinasen und ihrer Inhibitoren, sowie einiger apoptoseregulierender Gene mit RT-PCR, cDNA-Array und Westernblotanalyse nachweisen. Insgesamt ergaben sich anhand der Ergebnisse keine Hinweise auf eine p53-abhängige oder eine Todesrezeptor-vermittelte Apoptose in Paclitaxel-behandelten BON-1-Zellen. Stattdessen könnte die Paclitaxel-induzierte Heraufregulation von Caspase-9 und eingeschränkt auch von Bax auf einen mitochondrialen Apoptosepathway hinweisen. In dieser Arbeit wurde erstmals ein durch Paclitaxel ausgelöster G2/M-Arrest, die Induktion von Apoptose und ein synergistischer radiosensitivierender Effekt in einer neuroendokrinen Pankreastumorzellinie (Karzinoid) nachgewiesen. Aufgrund der in vitro gewonnenen Ergebnisse wäre daher ein verbessertes klinisches Ansprechen von neuroendokrinen pankreatischen Tumoren (Karzinoid) unter Paclitaxel-Behandlung in vivo denkbar

Chemoenzymatic Synthesis of Chromodepsipeptides and Natural Product Discovery via Genome Mining

Recent advances in the development of sequencing technologies have enabled the identification of a multitude of bacterial gene clusters, putatively involved in the biosynthesis of nonribosomal peptides (NRPs). Peptides of nonribosomal origin constitute a class of structurally and functionally diverse natural products, which are assembled by multimodular nonribosomal peptide synthetases (NRPSs). These compounds exhibit a broad pharmacological spectrum, ranging from antibacterial- to immunosuppressive properties. Understanding the assembly mechanisms in combination with rational genome mining approaches will provide opportunities for the discovery of new bioactive natural products. Within this study one approach was utilized to generate thiocoraline analogs via chemoenzymatic synthesis and the second strategy focused on the de novo natural product discovery via genome mining. Thiocoraline represents a pseudosymmetrical chromophore-capped octathiodepsipeptide, in which the symmetrical halves are linked via thioester bonds. In this study, the cyclodimerization potential of the thioesterase domain of the thiocoraline biosynthetic machinery (TioS PCP-TE) was investigated to obtain further insights into the iterative assembly of chromodepsipeptides. To address this objective, the recombinant enzyme was incubated with synthetically derived tetrapeptidyl substrates, resembling thiocoraline precursors. It was shown that the enzyme catalyzes the cyclodimerization of linear precursor molecules and an unprecedented macrothiolactonization. Evaluation of the biocombinatorial potential established the thioesterase as a robust and versatile catalyst for the generation of chromodepsipeptide analogs, harbouring thioester- or ester-linkages. As thiocoraline attains its antitumor activity from DNA-bisintercalation, the chemoenzymatically generated macrocycles were isolated and investigated towards DNA-bisintercalation activity in vitro. In the second part of this study, bioinformatic analysis of the 8.2 Mb Saccharopolyspora erythraea genome revealed two cryptic NRPS gene clusters related to hydroxamate-type siderophore biosynthesis. Detailed analysis of adenylation domain substrate-specificity and module organization enabled the establishment of a highly selective and sensitive radio-LCMS-guided genome mining approach. Application of this approach resulted in the discovery of the siderophore erythrochelin. Structure elucidation of erythrochelin was accomplished via NMR- and MSn-analysis and revealed the sequence of the tetrapeptide siderophore to be: α-N-acetyl-δ-N-acetyl-δ-N-hydroxy-D-ornithine-D-serine-cyclo(δ-N-hydroxy-L-ornithine-δ-N-acetyl-δ-N-hydroxy-L-ornithine). Erythrochelin assembly requires the proliferation of δ-N-hydroxy-L-ornithine (L-hOrn) and δ-N-acetyl-δ-N-hydroxy-L-ornithine (L-haOrn). The corresponding modifying enzymes, the FAD-dependent monooxygenases EtcB and Sace_1309 together with the bifunctional malonyl-CoA decarboxylase/N-acetyltransferase were identified and biochemically characterized. In vitro studies revealed EtcB and Sace_1309 to exclusively catalyze the δ-N-hydroxylation of free L-ornithine. The second tailoring enzyme, Mcd, was shown to catalyze malonyl-CoA decarboxylation and subsequent acetyltransfer onto the δ-hydroxamino group of L-hOrn, affording L-haOrn. Based on the elucidation of precursor biosynthesis (L-haOrn), a model for the entire erythrochelin assembly is presented

Exploiting Anonymity in Approximate Linear Programming: Scaling to Large Multiagent MDPs

The Markov Decision Process (MDP) framework is a versatile method for addressing single and multiagent sequential decision making problems. Many exact and approximate solution methods attempt to exploit struc- ture in the problem and are based on value factoriza- tion. Especially multiagent settings (MAS), however, are known to suffer from an exponential increase in value component sizes as interactions become denser, meaning that approximation architectures are overly re- stricted in the problem sizes and types they can handle. We present an approach to mitigate this limitation for certain types of MASs, exploiting a property that can be thought of as ‘anonymous influence’ in the factored MDP. In particular, we show how anonymity can lead to representational and computational efficiencies, both for general variable elimination in a factor graph but also for the approximate linear programming solution to factored MDPs. The latter allows to scale linear pro- gramming to factored MDPs that were previously un- solvable. Our results are shown for a disease control do- main over a graph with 50 nodes that are each connected with up to 15 neighbors

The MADP Toolbox: An Open-Source Library for Planning and Learning in (Multi-)Agent Systems

This article describes the MultiAgent Decision Process (MADP) toolbox, a software library to support planning and learning for intelligent agents and multiagent systems in un- certain environments. Some of its key features are that it sup- ports partially observable environments and stochastic tran- sition models; has unified support for single- and multiagent systems; provides a large number of models for decision- theoretic decision making, including one-shot decision mak- ing (e.g., Bayesian games) and sequential decision mak- ing under various assumptions of observability and coopera- tion, such as Dec-POMDPs and POSGs; provides tools and parsers to quickly prototype new problems; provides an ex- tensive range of planning and learning algorithms for single- and multiagent systems; and is written in C++ and designed to be extensible via the object-oriented paradigm

Effective Approximations for Spatial Task Allocation Problems

Although multi-robot systems have received substantial research attention in recent years, multi-robot coordination still remains a difficult task. Especially, when dealing with spatially distributed tasks and many robots, central control quickly becomes infeasible due to the exponential explosion in the number of joint actions and states. We propose a general algorithm that allows for distributed control, that overcomes the exponential growth in the number of joint actions by aggregating the effect of other agents in the system into a probabilistic model, called subjective approximations, and then choosing the best response. We show for a multi-robot grid-world how the algorithm can be implemented in the well studied Multiagent Markov Decision Process framework, as a sub-class called spatial task allocation problems (SPATAPs). In this framework, we show how to tackle SPATAPs using online, distributed planning by combining subjective agent approximations with restriction of attention to current tasks in the world. An empirical evaluation shows that the combination of both strategies allows to scale to very large problems, while providing near-optimal solutions

Effective Approximations for Multi-Robot Coordination in Spatially Distributed Tasks

Although multi-robot systems have received substantial research attention in recent years, multi-robot coordination still remains a difficult task. Especially, when dealing with spatially distributed tasks and many robots, central control quickly becomes infeasible due to the exponential explosion in the number of joint actions and states. We propose a general algorithm that allows for distributed control, that overcomes the exponential growth in the number of joint actions by aggregating the effect of other agents in the system into a probabilistic model, called subjective approximations, and then choosing the best response. We show for a multi-robot grid-world how the algorithm can be implemented in the well studied Multiagent Markov Decision Process framework, as a sub-class called spatial task allocation problems (SPATAPs). In this framework, we show how to tackle SPATAPs using online, distributed planning by combining subjective agent approximations with restriction of attention to current tasks in the world. An empirical evaluation shows that the combination of both strategies allows to scale to very large problems, while providing near-optimal solutions