What model does MuZero learn?

Model-based reinforcement learning has drawn considerable interest in recent years, given its promise to improve sample efficiency. Moreover, when using deep-learned models, it is potentially possible to learn compact models from complex sensor data. However, the effectiveness of these learned models, particularly their capacity to plan, i.e., to improve the current policy, remains unclear. In this work, we study MuZero, a well-known deep model-based reinforcement learning algorithm, and explore how far it achieves its learning objective of a value-equivalent model and how useful the learned models are for policy improvement. Amongst various other insights, we conclude that the model learned by MuZero cannot effectively generalize to evaluate unseen policies, which limits the extent to which we can additionally improve the current policy by planning with the model

Dynamics and thermodynamics of decay in charged clusters

We propose a method for quantifying charge-driven instabilities in clusters, based on equilibrium simulations under confinement at constant external pressure. This approach makes no assumptions about the mode of decay and allows different clusters to be compared on an equal footing. A comprehensive survey of stability in model clusters of 309 Lennard-Jones particles augmented with Coulomb interactions is presented. We proceed to examine dynamic signatures of instability, finding that rate constants for ejection of charged particles increase smoothly as a function of total charge with no sudden changes. For clusters where many particles carry charge, ejection of individual charges competes with a fission process that leads to more symmetric division of the cluster into large fragments. The rate constants for fission depend much more sensitively on total charge than those for ejection of individual particles

Semantic Knowledge-Based-Engineering: The Codex Framework

The development of complex systems within multi-domain environments requires an effective way of capturing, sharing and integrating knowledge of the involved experts. Modern Knowledge-Based Engineering (KBE) systems fulfill this function, making formalized knowledge executable by using highly specialized environments and languages. However, the dedication of these environments to their domain of application poses limitations on the cross-domain integration of KBE applications. The use of Semantic Web Technologies (SWT) delivers a domain-neutral way of knowledge formalization and data integration which promises to drastically reduce the effort required to integrate knowledge of multiple domains in a single representation. Especially within the complex field of aeronautical vehicle design the authors are working in, characterized by several individual disciplines having to be considered simultaneously, the combined usage of KBE and SWT technologies seems an attractive approach for the c ontinued digitalization of the design process. In this paper, the COllaborative DEsign and eXploration (Codex) framework is presented which aims at merging these two technologies into a single framework that can be used to create domain-specific knowledge-bases and integrate these into a single model of the overall product. Formalizing and executing this model will lead to a more transparent and integrated view on complex product design

Evaluation of the EndoPAT as a Tool to Assess Endothelial Function

Endothelial dysfunction is a potential target for (pharmaceutical) intervention of several systemic pathological conditions. We investigated the feasibility of the EndoPAT to evaluate acute changes in endothelial function with repeated noninvasive measurements and assessed its discriminating power in different populations. Endothelial function was stable over a longer period of time in renally impaired patients (coefficient of variation 13%). Endothelial function in renally impaired and type 2 diabetic patients was not decreased compared to healthy volunteers (2.9 ± 1.4 and 1.8 ± 0.3, resp., versus 1.8 ± 0.5, P > 0.05). The EndoPAT did not detect an effect of robust interventions on endothelial function in healthy volunteers (glucose load: change from baseline 0.08 ± 0.50, 95% confidence interval −0.44 to 0.60; smoking: change from baseline 0.49 ± 0.92, 95% confidence interval −0.47 to 1.46). This suggests that at present the EndoPAT might not be suitable to assess (changes in) endothelial function in early-phase clinical pharmacology studies. Endothelial function as measured by the EndoPAT could be physiologically different from endothelial function as measured by conventional techniques. This should be investigated carefully before the EndoPAT can be considered a useful tool in drug development or clinical practice

Critical assessment of human metabolic pathway databases: a stepping stone for future integration

<p>Abstract</p> <p>Background</p> <p>Multiple pathway databases are available that describe the human metabolic network and have proven their usefulness in many applications, ranging from the analysis and interpretation of high-throughput data to their use as a reference repository. However, so far the various human metabolic networks described by these databases have not been systematically compared and contrasted, nor has the extent to which they differ been quantified. For a researcher using these databases for particular analyses of human metabolism, it is crucial to know the extent of the differences in content and their underlying causes. Moreover, the outcomes of such a comparison are important for ongoing integration efforts.</p> <p>Results</p> <p>We compared the genes, EC numbers and reactions of five frequently used human metabolic pathway databases. The overlap is surprisingly low, especially on reaction level, where the databases agree on 3% of the 6968 reactions they have combined. Even for the well-established tricarboxylic acid cycle the databases agree on only 5 out of the 30 reactions in total. We identified the main causes for the lack of overlap. Importantly, the databases are partly complementary. Other explanations include the number of steps a conversion is described in and the number of possible alternative substrates listed. Missing metabolite identifiers and ambiguous names for metabolites also affect the comparison.</p> <p>Conclusions</p> <p>Our results show that each of the five networks compared provides us with a valuable piece of the puzzle of the complete reconstruction of the human metabolic network. To enable integration of the networks, next to a need for standardizing the metabolite names and identifiers, the conceptual differences between the databases should be resolved. Considerable manual intervention is required to reach the ultimate goal of a unified and biologically accurate model for studying the systems biology of human metabolism. Our comparison provides a stepping stone for such an endeavor.</p