Factorized Q-Learning for Large-Scale Multi-Agent Systems

Deep Q-learning has achieved significant success in single-agent decision making tasks. However, it is challenging to extend Q-learning to large-scale multi-agent scenarios, due to the explosion of action space resulting from the complex dynamics between the environment and the agents. In this paper, we propose to make the computation of multi-agent Q-learning tractable by treating the Q-function (w.r.t. state and joint-action) as a high-order high-dimensional tensor and then approximate it with factorized pairwise interactions. Furthermore, we utilize a composite deep neural network architecture for computing the factorized Q-function, share the model parameters among all the agents within the same group, and estimate the agents' optimal joint actions through a coordinate descent type algorithm. All these simplifications greatly reduce the model complexity and accelerate the learning process. Extensive experiments on two different multi-agent problems demonstrate the performance gain of our proposed approach in comparison with strong baselines, particularly when there are a large number of agents.Comment: 7 pages, 5 figures, DAI 201

Exploring the Impact of R&D on Patenting Activity in Small Women-Owned and Minority-Owned Entrepreneurial Firms.

The relevant economics literature on the impact of R&D on patenting activity falls within two methodological areas of inquiry. The first area might be classified as a test of the Schumpeterian hypothesis. The second and lesser research area might be classified as an estimation of the knowledge production function relationship between R&D and patenting. This paper focuses on estimates of the R&D-to-patenting relationship for a random sample of small, entrepreneurial firms whose research projects were supported through the US Small Business Innovation Research (SBIR) program. Our paper contributes to the R&D-to-patenting literature in two ways. It examines empirically a unique set of small, entrepreneurial firms funded by the public sector, and it explores the effect of the gender and ethnicity of firm owners on the propensity of their firms to patent from funded research projects

Knowledge begets knowledge: university knowledge spillovers and the output of scientific papers from U.S. Small Business Innovation Research (SBIR) projects

Scientific papers submitted for publication from U.S. Small Business Innovation Research (SBIR)-funded research projects are an innovative output that has yet to be studied systematically. Using a knowledge production framework, we identify empirically covariates with the number of scientific papers resulting from SBIR projects over the period 1992 through 2001. We find empirically that when the firm involves a university in its funded project, more scientific papers result. When the form of university involvement is taken into account, we find the greatest impact on the output of scientific papers comes from the inclusion of an individual from the university who originally developed the technology being pursued by the firm in its SBIR project. In other words, the project-specific technical human capital knowledge from the university that spills over to the firm’s project begets (i.e., brings about) additional knowledge in the form of scientific papers submitted for publication

The impact of public R&D investments on patenting activity: technology transfer at the U.S. Environmental Protection Agency.

This paper presents estimates of the impact of public R&D on patenting activity at the U.S. Environmental Protection Agency (EPA). Using a time series of public sector agency data, we estimate the per-capita R&D elasticity of new patent applications using a knowledge production function framework model that is an expanded version of what other scholars have used with private sector data. New patent applications are an important step in the technology transfer activities of a federal agency. We estimate this elasticity to be about 2.0. This elasticity value represents an initial estimate of the impact of EPA’s R&D investments on its technology transfer activity

Intracranial bleeding due to vitamin K deficiency: advantages of using a pediatric intensive care registry

Item does not contain fulltextAIM: To determine the incidence of late intracranial vitamin K deficiency bleeding (VKDB) in The Netherlands using the Dutch Pediatric Intensive Care Evaluation (PICE) registry. METHODS: The PICE registry was used to identify all infants who were admitted to a Dutch pediatric intensive care unit (PICU) with intracranial bleeding between 1 January 2004 and 31 December 2007. Cases of confirmed late intracranial VKDB were used to calculate the incidence for each year. To estimate the completeness of ascertainment of the PICE registry, data from 2005 were compared with general surveillance data from that year. RESULTS: In the 4-year study period, 16/64 (25%) of the infants admitted with intracranial bleeding had late intracranial VKDB, resulting in an overall incidence of 2.1/100,000 live births (95% confidence interval 1.2-3.5). The single-year incidence varied markedly between 0.5 and 3.3 per 100,000 live births. All five ascertained cases in 2005 were identified using the PICE registry, while general surveillance identified only three. CONCLUSIONS: The PICE registry allows ongoing monitoring of the incidence of late intracranial VKDB and appears to be associated with a higher rate of completeness than general surveillance. We propose the use of pediatric intensive care registries to assess the efficacy of national vitamin K prophylactic regimens

A public sector knowledge production function

There are no studies of the R&D-to-patenting relationship at the federal agency level. We estimate a public sector knowledge production function using federal agency patent application data over the years 2003 through 2014. We find that the patent application elasticity with respect to per capita R&D spending is about 1.06. This measure might be interpreted as one dimension of the social returns to public sector R&D generated through newly created knowledge

The Use of Intellectual Property Protection Mechanisms by Publicly Supported Firms

Technology-based firms use intellectual property protection mechanisms (IPPMs) to appropriate the returns to their research investments. The empirical literature has generally focused on the use of IPPMs by private sector firms to appropriate the returns to their privately financed R&D-based technologies. To date, studies have not considered the use of IPPMs by private sector firms whose research is publicly financed. We identify empirically a number of significant covariates with the use of a portfolio of formal IPPMs consisting of patents, copyrights, and trademarks. However, our multivariate empirical analyses show that caution is needed in generalizing about such covariates when discussing any one particular formal IPPM