7,583 research outputs found

    Reinforcement learning in large state action spaces

    Get PDF
    Reinforcement learning (RL) is a promising framework for training intelligent agents which learn to optimize long term utility by directly interacting with the environment. Creating RL methods which scale to large state-action spaces is a critical problem towards ensuring real world deployment of RL systems. However, several challenges limit the applicability of RL to large scale settings. These include difficulties with exploration, low sample efficiency, computational intractability, task constraints like decentralization and lack of guarantees about important properties like performance, generalization and robustness in potentially unseen scenarios. This thesis is motivated towards bridging the aforementioned gap. We propose several principled algorithms and frameworks for studying and addressing the above challenges RL. The proposed methods cover a wide range of RL settings (single and multi-agent systems (MAS) with all the variations in the latter, prediction and control, model-based and model-free methods, value-based and policy-based methods). In this work we propose the first results on several different problems: e.g. tensorization of the Bellman equation which allows exponential sample efficiency gains (Chapter 4), provable suboptimality arising from structural constraints in MAS(Chapter 3), combinatorial generalization results in cooperative MAS(Chapter 5), generalization results on observation shifts(Chapter 7), learning deterministic policies in a probabilistic RL framework(Chapter 6). Our algorithms exhibit provably enhanced performance and sample efficiency along with better scalability. Additionally, we also shed light on generalization aspects of the agents under different frameworks. These properties have been been driven by the use of several advanced tools (e.g. statistical machine learning, state abstraction, variational inference, tensor theory). In summary, the contributions in this thesis significantly advance progress towards making RL agents ready for large scale, real world applications

    Generalizations for Cell Biological Explanations: Distinguishing between Principles and Laws

    Get PDF
    Laws have figured in the development of modern biology (e.g. Mendelian laws of inheritance), but there is a tacit assumption particularly in contemporary cell and molecular biology that laws are only of the 'strict' kind (e.g. the laws of motion or universal gravitation), which cell biology appears to lack. Moreover, the cell-biology-specific non-universal laws that do exist (e.g. scaling laws in biochemical networks within single cells) are few and far between. As discussed elsewhere (and not further argued for in this paper), mechanistic explanations, which are the dominant kind of explanation in cell biology, face significant challenges and their utility has been checkered in different biomedical areas. Just as laws and mechanisms figure in explanations in organic chemistry and ecology, fields that deal with lower- and higher-scale phenomena compared to cell biology, respectively, it should not be assumed that cell biology is somehow in a unique position where few or no laws could be discovered and used in its explanations. An impediment to discovering lawlike generalizations in cell biology is that the understanding of many cellular phenomena is still quite qualitative and imprecise. This paper is motivated by the premise that mechanisms and laws can both be in the foreground of explanations in cell biology and that a framework should be developed to encourage and facilitate the discovery of laws specific to and operative at the individual cell level. To that end, in the domain of scientifically-relevant non-universal (i.e. non-exceptionless) generalizations, which some philosophers equate with the notion of ceteris paribus laws (henceforth, 'cp-laws'), I propose that a cp-law might have one or more corresponding 'principles'. Using a running example of generalizations of oscillatory movements from physics with direct relevance to cell biology, I argue that while a cp-law and its paired principle(s) might have the same explanatory theme (e.g. explain the same phenomenon), a principle is broader in scope compared to its paired cp-law but less expectable or reliable in its predictions. This is because principles appear to be more qualitative and less numerically precise compared to cp-laws, reflective of our lack of precise understanding of the systems to which the generalizations apply. The principlesā€“laws concept makes for a more lenient approach for what could count as a lawlike generalization and can encourage the discovery of novel generalizations in areas of cell biology where no specific generalizations typically figure in explanations. A principle could be thought of as providing a program for explanation, whereas its paired law provides explanations for specific instances. Newly posited principles could augment mechanistic explanations and also potentially lead to the discovery of corresponding cp-laws

    BIOTECHNOLOGY PATENT LAW TOP TEN OF 2021. EXPERIMENTATION, BLAZE MARKS, AND UNSPECIFIED RANGES

    Get PDF
    Biotechnology has never demonstrated its benefits to society more than in 2021. The SARS-CoV-2 virus that caused the CoVID-19 pandemic met a formidable opponent in mRNA vaccines developed and supplied by Moderna and Pfizer/BioNTech. These vaccines are claimed in myriad ā€“ not Myriad ā€“ patents and patent applications, many of which are destined to be litigated over the coming years, not least inspired by the many billions of dollars that have been, and will continue to be, earned by their owners. While the world waits for this storm of patent litigation, federal courts continue to be busy with ownership, licensing, validity, and infringement disputes arising from other biotechnologies, including, perhaps, up-and-coming CAR-T therapies. For the fourth year in a row (of what has become a tradition), we discuss, in this article, the ten most consequential, important, and interesting court decisions involving biotechnology patents. Our top ten decisions may not be the same as top tens compiled by others. However, to quote an expression commonly heard in courts hearing patent cases, Ć  chacun son goĆ»t. Patent decisions delivered during 2021 tackled a diverse group of doctrinal issues. As discussed in the article, these ranged from how much experimentation is to be considered undue Ć  la In re Wands, to what level of detail of disclosure is sufficient to satisfy the ever- written description requirement, to which types of behavior may rise to the level of inducement to infringe, not to mention assignor estoppel. Patent litigations filed in federal district court rose to 3,798, a number not seen since 2016. In contrast, the 1,333 patent actions filed with the Patent Trial and Appeal Board (ā€œPTABā€œ) represented a substantial decline from 2020. In short, despite the challenges of the CoVID- 19 pandemic, patent litigation in 2021 evinced considerable vim and vigor. Described and analyzed in this article are the vimmiest and most vigorous of 2021 patent decisions

    Limit theorems for non-Markovian and fractional processes

    Get PDF
    This thesis examines various non-Markovian and fractional processes---rough volatility models, stochastic Volterra equations, Wiener chaos expansions---through the prism of asymptotic analysis. Stochastic Volterra systems serve as a conducive framework encompassing most rough volatility models used in mathematical finance. In Chapter 2, we provide a unified treatment of pathwise large and moderate deviations principles for a general class of multidimensional stochastic Volterra equations with singular kernels, not necessarily of convolution form. Our methodology is based on the weak convergence approach by Budhiraja, Dupuis and Ellis. This powerful approach also enables us to investigate the pathwise large deviations of families of white noise functionals characterised by their Wiener chaos expansion as~XĪµ=āˆ‘n=0āˆžĪµnIn(fnĪµ).X^\varepsilon = \sum_{n=0}^{\infty} \varepsilon^n I_n \big(f_n^{\varepsilon} \big). In Chapter 3, we provide sufficient conditions for the large deviations principle to hold in path space, thereby refreshing a problem left open By PĆ©rez-Abreu (1993). Hinging on analysis on Wiener space, the proof involves describing, controlling and identifying the limit of perturbed multiple stochastic integrals. In Chapter 4, we come back to mathematical finance via the route of Malliavin calculus. We present explicit small-time formulae for the at-the-money implied volatility, skew and curvature in a large class of models, including rough volatility models and their multi-factor versions. Our general setup encompasses both European options on a stock and VIX options. In particular, we develop a detailed analysis of the two-factor rough Bergomi model. Finally, in Chapter 5, we consider the large-time behaviour of affine stochastic Volterra equations, an under-developed area in the absence of Markovianity. We leverage on a measure-valued Markovian lift introduced by Cuchiero and Teichmann and the associated notion of generalised Feller property. This setting allows us to prove the existence of an invariant measure for the lift and hence of a stationary distribution for the affine Volterra process, featuring in the rough Heston model.Open Acces

    Combinatorics and Stochasticity for Chemical Reaction Networks

    Get PDF
    Stochastic chemical reaction networks (SCRNs) are a mathematical model which serves as a first approximation to ensembles of interacting molecules. SCRNs approximate such mixtures as always being well-mixed and consisting of a finite number of molecules, and describe their probabilistic evolution according to the law of mass-action. In this thesis, we attempt to develop a mathematical formalism based on formal power series for defining and analyzing SCRNs that was inspired by two different questions. The first question relates to the equilibrium states of systems of polymerization. Formal power series methods in this case allow us to tame the combinatorial complexity of polymer configurations as well as the infinite state space of possible mixture states. Chapter 1 presents an application of these methods to a model of polymerizing scaffolds. The second question relates to the expressive power of SCRNs as generators of stochasticity. In Chapter 2, we show that SCRNs are universal approximators of discrete distributions, even when only allowing for systems with detailed-balance. We further show that SCRNs can exactly simulate Boltzmann machines. In Chapter 3, we develop a formalism for defining the semantics of SCRNs in terms of formal power series which grew as a result of work included in the previous chapters. We use that formulation to derive expressions for the dynamics and stationary states of SCRNs. Finally, we focus on systems that satisfy complex balance and conservation of mass and derive a general expressions for their factorial moments using generating function methods

    At the intersection between machine learning and econometrics: theory and applications

    Get PDF
    In the present work, we introduce theoretical and application novelties at the intersection between machine learning and econometrics in social and health sciences. In particular, Part 1 delves into optimizing the data collection process in a specific statistical model, commonly used in econometrics, employing an optimization criterion inspired by machine learning, namely, the generalization error conditioned on the training input data. In the first Chapter, we analyze and optimize the trade-off between sample size, the precision of supervision on a variation of the unbalanced fixed effects panel data model. In the second Chapter we extend the analysis to the Fixed Effects GLS (FEGLS) case in order to account for the heterogeneity in the data associated with different units, for which correlated measurement errors corrupt distinct observations related to the same unit. In Part 2, we introduce\ud applications of innovative econometrics and machine learning techniques. In the third Chapter we propose a novel methodology to explore the effect of market size on market innovation in the Pharmaceutical industry. Finally, in the fourth Chapter, we innovate the literature on the economic complexity of countries through machine learning. The Dissertation contributes to the literature on machine learning and applied econometrics mainly by: (i) extending the current framework to novel scenarios and applications (Chapter 1 - Chapter 2); (ii) developing a novel econometric methodology to assess long-debated issues in literature (Chapter 3); (iii) constructing a novel index of economic complexity through machine learning (Chapter 4)

    The historical consciousness of eighteenth-century Britain: Viscount Bolingbroke and Edmund Burke

    Get PDF
    How did eighteenth-century Britain deal with the tension between its traditional political and religious foundations and the rise of commercial society with its emphases on individual self-interest and the accumulation of wealth? This thesis explores one major response to the tension ā€“ the effort to discern a historical consciousness that would help to understand change and to preserve the social order. It focuses on two major conservative thinkers: Viscount Bolingbroke (1678-1751) and Edmund Burke (1729-1797), and it shows how their different understandings of the historical consciousness shaped their attitudes toward the emerging commercial society, individual self-interest, the expansion of empire, and political reform movements. The thesis maintains that Bolingbroke and Burke represented two distinct versions of historical consciousness. Bolingbrokeā€™s historical consciousness was characterized by a linear view of history which considered the past, present and future as separate spaces of time. The thesis shows how Bolingbrokeā€™s rejection of the political management and promotion of commercial interests represented by Sir Robert Walpoleā€™s Whig party was based on an historical consciousness that idealized Englandā€™s ancient constitution and traditional social morality, and was motivated by a patriotic effort to restore a past golden age. The thesis also shows how Bolingbrokeā€™s historical consciousness reflected his Deist religious beliefs, his notion of natural law, and his rationalist philosophical approaches. The thesis shows how Bolingbrokeā€™s radical historical consciousness was later taken up by French Enlightenment philosophers and English Dissenters ā€“ who in turn developed views of social progress based on natural rights theory. Edmund Burke, on the other hand, saw a threat to the existing order coming from these emphases on natural law and natural rights. Burkeā€™s historical consciousness confronted Bolingbroke and his successors by taking fundamentally different positions on the origin and formation of civil order, on continuity and change of society, and on the relationship between divine will, human reason and history. Burkeā€™s historical consciousness assumed that society was a living partnership between different generations. The past, the present and the future were not separate spaces of time but co-existed in the same space of time, and society was constantly in a changeable state, as a union of the principles of ā€œrenovationā€, ā€œpermanenceā€ and ā€œchangeā€. Burke rejected Bolingbrokeā€™s idea of a past golden age and an ancient constitution. For Burke, society and the state were offspring of social conventions and human history, rather than of natural law and natural rights. Moreover, Burke conceived human history as unpredictable, shaped by uncertain and obscure factors. There was, for Burke, no ultimate cause or general rule determining the course of history. The thesis concludes that Burke understood the history of human society to be a process that transcended any systematic design that could be discerned by human reason alone

    Influence diagrams for complex litigation

    Get PDF
    Effective advocacy depends critically on the ability of attorneys to formulate, analyze, and compare rival courses of action. Whereas attorneys have been doing these things for centuries using little more than their gut instincts and experiences, sophisticated decision aids are now available that can improve the way attorneys assess the value of their cases and the strategic decisions that they make. These aids are proving valuable in medicine and business, but they have not impacted legal practice. This Article seeks to correct this oversight by showing how easy-to-use graphical models provide guidance for strategic legal decisions. Beginning with a paradigmatic example of a plain- tiff who must choose between proceeding to trial or settling out of court, the Article shows how decision aids handle the uncertainties and interdependencies that arise when real-world considerations are introduced. In particular, the Article makes the case that influence diagrams, a relative newcomer in the field of decision analysis, should be the decision aid of choice in complex litigation matters

    Understanding Your Game: A Mathematician's Advice for Rational and Safe Gambling

    Get PDF
    The author proposes in this practical guide for both problem and non-problem gamblers a new pragmatic, conceptual approach of gambling mathematics. The primary aim of this guide is the adequate understanding of the essence and complexity of gambling through its mathematical dimension. The author starts from the premise that formal gambling mathematics, which is hardly even digestible for the non-math-inclined gamblers, is ineffective alone in correcting the specific cognitive distortions associated with gambling. By applying the latest research results in this field, the author blends the gambling-mathematics concepts with the epistemology of applied mathematics and cognitive psychology for providing gamblers the knowledge required for rational and safe gambling
    • ā€¦
    corecore